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INTRODUCTION 


The  training  of  the  individual  soldier  as  well  as  collective  training 
has,  in  recent  years,  become  a  more  challenging  and  difficult  task.  The 
dwindling  resources,  the  competing  demands  for  time  and  the  more  complex 
tasks  to  be  trained  that  are  presently  being  experienced  by  Army  person¬ 
nel  Increases  the  need  to  develop  the  most  efficient  and  effective 
methods  of  training  possible.  In  response  to  these  demands  and  require¬ 
ments,  the  Army  Training  Study  (ARTS)  was  initiated  to  explore  means  of 
developing  an  "efficient,  justifiable,  and  achievable  training  system 
for  the  Army  of  the  mid-1980's,"^  ARTS’  approach  to  the  problem  was  to 
formulate  both  a  short  term  and  long  term  effort.  The  short  term  effort, 
the  Training  Effectiveness  Analysis  (TEA)  78,  was  designed  to  capitalize 
on  selected  presently  on-going  training  development  and  evaluation  pro¬ 
grams  as  a  means  to  economically  glean  as  much  early  information  as 
possible  and  for  deriving  insights  and  direction  for  the  longer  term 
TEA  85  effort.  TEA  85  is  aimed  at  quantification  of  current  training 
systems  in  order  to  support  allocations  of  required  training  resources 
and  as  a  basis  for  evolving  training  system  improvements  designed  to 
provide  the  Army  with  the  most  efficient  and  effective  training  systems. 

The  Combined  Arms  Center  (CAC)  as  proponent  for  all  TRADOC  battle 
simulations  is  presently  involved  in  th^  development  of  several  collec¬ 
tive  training  programs  for  command  groups  at  battalion  and  above  levels. 

^DA  Msg,  ATCG-ATS ,  DTG  221832Z  Nov  77,  Subj  :  Army  Training  Study 
(ARTS)  . 
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At  Che  request  of  ARTS,  Chese  programs  were  examined  co  ascertain  if 
any  "piggy-backing"  were  possible  Co  exploit  this  unique  source  of  com¬ 
mand  group  training  data.  There  were  two  major  areas  of  continuing 
concern  in  which  an  integration  between  the  developmental  effort  for 
training  command  groups  and  the  ARTS  effort  appeared  feasible: 

1.  What  is  the  relationship  among  command  group  performance  as 
assessed  in  battle  simulation,  unit  readlnesa,  and  combat  effectiveness 
measures? 

2.  What  are  appropriate  strategies  to  achieve  optimum  (cost  and 
training  effectiveness)  command  group  proficiency  levels  through  use 
of  simulation  technology? 

Obviously  definitive  answers  to  these  questions  would  not  be  pos¬ 
sible  for  ARTS  78  within  the  cime,  resource  and  technology  constraints 
prevailing.  However,  the  need  for  and  mutual  interest  in  even  tenta¬ 
tive  answers  to  portions  of  the  above  two  questions  prompted  the  initia¬ 
tion  of  an  exploratory  effort  utilizing  a  current  training  system 
undergoing  development  by  CAC:  the  Computer  Assisted  Map  Maneuver 
System  (CAMMS) .  It  was  determined  that  only  five  battalion  command 
groups  from  two  divisions  were  available  for  participation  in  this 
effort  in  time  for  input  into  the  TEA  78  Report.  It  was  recognized 
that  the  small  sample  size  and  limited  representativeness  would  restrict 
generalizabillty  and  any  analysis  would  therefore  be  of  value  mainly  in 
terms  of  preliminary  indications,  trends  and  feasibility  of  approach. 
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However,  it  was  felt  that  an  initial  look  at  command  group  training  was 
necessary  and  should  provide  a  source  of  useful  planning  information 
for  ARTS  as  well  as  CAC. 

OBJECTIVES 

Specific  objectives  of  this  command  group  training  effort  derive 
from  the  broad  questions  identified  above  and  the  problem  of  measure¬ 
ment  Inherent  in  achieving  answers  to  those  questions.  Only  those 
objectives  of  direct  relevance  to  the  ARTS  which  could  be  addressed 
within  the  tine  constraints  of  TEA  78  are  Included  below.  Other  objec¬ 
tives  and  analyses  will  be  the  subject  of  subsequent  separate  documenta¬ 
tion. 

Specifically,  the  present  effort  focused  on  three  objectives: 

f  ‘  *  A  < :  |  /-  ...  r  .  s  r .  ?  *  $  '  ~- 

1.  To  estimate  the  training  effectiveness  of, CAMMS.  This  objective 
was  aimed  at  the  type  and  amount  of  performance  Improvement  that  can  be 
produced  with  a  battle  simulation  (i.e.,  CAMMS) .  Implicit  in  this 
objective  is  the  development  of  command  group  performance  assessment 
procedures  and  feedback  mechanisms  that  would  allow  the  command  group  to 
diagnose  their  training  strengths  and  weaknesses ,and  would  enable  meaning¬ 
ful  feedback  to  the  command  group  during  and  between  training  sessions  so 
that  their  training  effort  could  be  concentrated  in  those  areas  where 
remedial  training  is  most  warranted. 

^  2.  To  refine  performance  measurement  procedures.  This  objective 

t 

was  designed  to  assist  in  the  development  of  more  valid  and  sensitive 
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.•>  means  of  measuring  Che  proficiency  level  of  battalion  command  groups. 

The  comnand  group/staff  module  of  AkTEP  71-2  is  a  necessary  though 
probably  not  sufficient  component  for  such  assessment.  In  addition, 
the  degree  to  which  measures  for  the  performance  of  various  of  the  tasks 
and  subcasks  could  be  made  more  objective  and  other  measures  of  a  more 
objective  nature  developed,  the  more  uniform,  equitable,  and  meaningful 
the  assessment  process  and  the  greater  Che  likelihood  of  achieving  a 
basis  for  comparing  performance  across  different  battle  simulations.. 

3.  To  estimate  the  feasibility  of  continuing  to  utilize  CAMMS  as  a 
vehicle  for  Investigating  command  group  training.  An  important  objective 
of  this  effort  was  to  determine  if  CAMMS  has  the  sensitivity  and  capa¬ 
bility  required  of  a  research  tool  for  providing  sufficient  data  to 
answer  some  of  the  unanswered  questions  associated  with  command  group 
training  and  to  ascertain  what  might  be  necessary  to  enhance  its  utility 
for  that  purpose. 


METHOD 


PARTICIPANTS 

Five  battalion  command  groups,  two  mechanized  infantry  and  three 
armor,  participated  in  this  effort.  These  groups  were  drawn  from  four 
brigades  within  two  CONUS  divisions. 

SIMULATION  SYSTEM 

CAMMS,  ^  as  mentioned  previously,  is  a  training  system  undergoing 
development  at  the  CAC.  The  system  is  being  designed  to  overcome  defi¬ 
ciencies  of  conventional  methods,  e.g.,  CPX,  FTX,  which  have  been  used 
to  provide  command  group  training.  It  is  a  battle  simulation  designed 
to  train  commanders  and  staffs  of  armor,  mechanized  infantry,  light 
infantry  and  cavalry  units  at  both  the  brigade  and  battalion  command 
levels.  The  command  groups  play  within  a  non-nuclear  environment  and 
against  a  given  enemy  force. 

CAMMS  served  as  the  instrument  by  which  data  on  the  performance  of 
the  aoovementioned  battalion  command  groups  was  obtained.  A  preliminary 
evaluation  of  the  training  effectiveness  of  the  CAMMS  was  implicit  in 
this  process.  For  the  purposes  of  this  effort,  the  simulation  was  con¬ 
ceived  to  consist  of  three  components.  How  each  of  these  components 
was  played  or  used  in  this  effort,  which  parallels  how  the  system  is 
normally  employed,  is  provided  below: 

^A  more  detailed  description  of  CAMMS  can  be  found  in  Battle  Simula¬ 
tions  and  the  ARTEP,  CATRADA,  Fort  Leavenworth,  Kansas,  November  1977. 
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The  Command  Group 


The  cocaumri  groups  were  constituted  of  those  persons  which  normally 
would  have  been  present  under  combat  conditions.  In  addition  to  the 
commander  and  principal  coordinating  staff  members,  the  groups  included 
the  Air  Force  liaison  officer,  the  fire  support  officer  and  supporting 
NCO's  and  RIO* a.  The  exercise  was  played  within  a  simulated  Tactical 
Operations  Center  (TOC)  equipped  with  the  type  of  communications  geer 
normally  Issued  to  the  battalion.  Thus,  the  command  groups  had  the 
capability  to  communicate  with  both  their  superior  and  subordinate  units 
as  well  as  adjacent  units  if  such  adjacent  unit-*  were  played. 

Controllers 

A  number  of  controllers,  whose  primary  purpose  was  to  manage  the  exer¬ 
cise  In  such  a  way  as  to  maximize -the  command  group  training  experience, 
were  used  to  conduct  the  exercise.  This  group  Included:  (1)  a  chief 
controller  who  played  the  role  of  the  brigade  commander;  (2)  a  brigade 
S1/S4  controller;  (3)  brigade  S2  and  brigade  S3  controllers;  (4)  three 
maneuver  company  commander  controllers;  (5)  a  fire  support  controller  and 
two  to  three  supporting  forward  observers;  (6)  a  direct  air  support  con¬ 
troller;  and  (7)  an  opposing  force  (OPFOR)  controller.  All  controllers, 
except  those  playing  the  brigade  commander  and  principal  staff,  maneuvered 
"troops"  and  items  of  simulated  equipment  appropriate  to  their  role  on  a 
game  board  that  served  as  the  terrain  on  which  the  battle  was  fought. 

Based  upon  the  results  of  calculations  produced  by  supporting  computer 
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software  and  the  events  which  were  unfolding  or  the  game  board  as  the 
battle  progressed,  both  the  game  board  and  brigaae  controllers  provided 
realistic  feedback  and  guidance  to  the  command  group  players.  An  adjunct 
member  of  the  controller  team  was  a  TOC  monitor  who  observed  the  activi¬ 
ties  of  the  command  group  during  planning  and  operational  phases  for  the 
purposes  of  providing  post  exercise  feedback  and  ratings  of  the  groups' 
behavior  for  analysis  in  this  investigation. 

Computer  Subsystem 

The  computer  subsystem  was  designed  to  support  military  and  logistics 
problems,  greatly  reduce  map  maneuver  preparation  time,  provide  faster 
and  more  accurate  computations  and,  thereby,  increase  objectivity  and 
provide  a  precise  summary  of  the  events  which  took  place  in  the  battle 
for  analysis  and  crltlqua  purposes.  The  software  available  accommodated 
the  employment  of  conventional  forces  viih  all  their  normal  supporting 
weapons  systems.  Artillery,  air,  mortars,  helicopters  and  admin/log 
functions  were  processed  for  the  friendly  force  as  well  as  the  OPFOR. 

The  system  allowed  for  task  organizations  ranging  from  specific  teams 
up  to  task  force  level.  Interface  with  these  programs  was  accomplished 
remotely  through  four  computer  terminals  which  were  connected  via  com- 
merldal  telephone  lines  to  a  centrally  located  computer.  The  terminal 
operators,  normally  vadio-teletype  operators,  were  provided  by  the 
participating  units . 
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DESIGN 


To  explicitly  address  the  first  and  implicitly  address  the  second  and 
third  objectives  of  this  effort,  a  pre-test/post-test  design  was  used. 
Figure  1  depicts  this  design.  The  command  groups  participated  in  three 
separate  exercises.  However,  the  general  scenario  and  type  of  mission 
were  the  same  across  the  three  exercises.  The  specific  scenario  and 
mission  used  for  each  exercise  was  a  variation  of  the  more  general  one. 
Also,  the  three  specific  scenarios  and  missions  were  designed  and  assumed 
to  be  of  equal  difficulty.  However,  to  correct  for  any  differences  in 
difficulty  that  may  have  existed,  the  missions  for  the  pre-  and  post¬ 
test  exercises  were  counterbalanced  across  units.  A  feedback  session, 
the  format  and  content  of  which  were  being  pilot  tested  in  this  effort, 
followed  each  of  the  test  and  training  exercises.  Ideally,  a  control 
group  which  received  only  the  pre-  and  post-tests  would  have  been  included 
to  verlf/  chat  those  performance  gains  observed,  if  any,  were  due  to  the 
treatment  or  training  rather  than  to  other  potential  confounding  variables. 
Additionally,  it  would  have  been  desirable  to  have  used  CAMMS  only  for  the 
treatment  condition  rather  than  across  all  conditions.  This  would  have 
made  it  possible  to  avoid  spurious  relations  that  may  be  inherent  in  the 
results  presented  herein  because  the  same  measuring  instrument  was  used 
both  for  testing  and  training  purposes.  Time  and  the  availability  of 
command  groups  precluded  fulfilling  either  of  these  two  conditions.  These 

^A  brief  description  cf  the  general  and  special  situations  for  all 
exercises  and  the  specific  mission  for  each  exercise  is  provided  in 
Appendix  A. 
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shortcomings  as  well  as  the  size  of  the  sample  and  instability  within 


the  controller  group,  a  problem  discussed  more  fully  below,  limit  sub¬ 
stantially  the  confidence  which  can  be  placed  in  the  results  presented 
in  following  sections. 

BATTALION  COMMAND  GROUP  PERFORMANCE  MEASURES 

A  number  of  different  types  of  performances  were  measured.  These 
measures  can  be  broken  down  into  two  broad  categories;  those  which  were 
obtained  through  a  rating  process  and  which,  therefore,  are  subjective  in 
nature  and  those  which  are  objective  in  nature.  Within  both  of  these 
categories,  performance  measures  related  to  a  selected  subset  of  the 
subtasks  contained  within  the  battalion/staff  module  of  ARTEP  71-2^  were 
gathered.  Of  the  61  subtasks  included  in  the  ARTEP,  47  were  measured,-* 
either  subjectively,  objectively  or  both.  Appendix  B  identifies  these 
47  subtasks.  Within  the  subjective  category  of  measures  were  also 
included  organizational  process  variables  which  have  previously  been 
used  in  the  investigation  of  unit  effectiveness.  A  more  thorough  dis¬ 
cussion  of  the  subjective  and  objective  measures,  in  terms  of  the  source 
from  which  they  were  obtained  and  the  rationale  for  their  investigation, 
is  provided  below. 

^Army  Training  and  Evaluation  Program  (ARTEP)  for  Mechanized  Infantry/ 
Tank  Task  Force,  No.  71-2,  Headquarters,  Department  of  the  Armv,  Washing¬ 
ton,  D.C.,  17  June  1977. 

^Performance  of  the  remaining  14  subtasks  was  not  measured  either 
because  these  performances  could  not  be  observed  or  because  thev  could 
not  readily  or  realistically  be  elicited. 
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Subjective  Measures 


ARTEP  Related.  Ratings  on  each  of  the  47  subtasks  wer  obtained 
from  one  or  more  members  of  the  control  team.  The  subtasks  rated  by 
each  of  the  controllers  who  provided  such  information  is  summarized 
below: 


Controller  Subtask  Rated 


Brigade  S1/S4 

3J, 

3K, 

9A, 

9B, 

9C, 

9D 

Brigade  S2 

IB, 

2A, 

2B, 

2C, 

2D, 

3F, 

31, 

5A,  5B, 

5C, 

5D, 

10A, 

,  12A 

Brigade  S3 

1A, 

1C, 

ID, 

IB, 

1H, 

3A, 

3B, 

3C,  3P, 

3G, 

3H, 

4B, 

6A, 

6B, 

7C, 

8A, 

8C, 

8D, 

10F, 

,  10G 

i,  11A 

FSO 

11, 

u. 

IK, 

1L, 

3E, 

7A, 

7B 

TOC  Observer  All  sub  tasks,  except  9A  to  9D. 

The  controllers  were  instructed  to  base  their  ratings  upon  the  conditions 
and  standards  stated  in  the  ARTEP  for  each  of  the  subtasks.  Their  ratings 
were  provided  on  five-point  Likert  type  scales  with  "forgotten,  overlooked 
or  unit  failed  to  address  this  subtask"  and  "exceeded  standard"  serving 
as  anchors  on  the  low  (1)  and  high  (5)  ends  of  the  scale  respectively. 

Two  complete  sets  of  ratings  were  obtained.  One  nearly  complete  set  was 
provided  by  the  TOC  monitor  and  the  second  set  was  provided  by  the  various 
controllers  who  responded  to  those  subtasks  which  were  within  their  area 
of  staff  responsibility.  These  two  sets  of  ratings  were  averaged  for 
analysis  purposes. 

In  addition  to  ratings  for  each  subt*''-;,  controllers  provided  overall 
estimates  of  how  well  the  command  group  performed  across  all  subtasks. 
Specifically,  overall  ratings  of  the  performance  of  the  SI,  S2,  S3,  S4, 

FSO  and  ALO,  battalion  commander  and  command  group  as  a  whole  were  obtained 
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from  Che  chief  controller  and  TOC  monitor.  Again,  a  five-point  scale 
with  low  and  high  anchors  respectively  being  "not  effective  at  all"  and 
"extremely  effective"  was  used.  The  TOC  observer  and  the  chief  con¬ 
troller  were  the  only  sources  from  which  these  overall  estimates  could 
be  obtained.  Averages  of  their  ratings  provided  the  raw  data  for  analysis. 

Organizational  Processes.  Olmstead,  et  al.  ^1973)  has  found  within  a 
military  context  that  the  ability  of  an  organization  to  cope  effectively 
with  its  environment  is  in  large  part  a  function  of  how  well  it  manages  to 
perform  certain  cask  clusters  or  processes  which  Schein  (1972)  had  identi¬ 
fied  in  earlier  theoretical  work.  These  processes  and  the  definition  of 
them  used  in  this  effort  are  presented  below: 

•  Acquiring  Information.  The  process  by  which  the  comaand  group 
acquired  information  about  its  external  and  internal  environment. 

e  Communicating  Information.  The  process  of  transmitting  information 
that  was  acquired  to  those  parts  of  the  unit  that  could  act  upon  it. 

e  Decision  Making.  The  process  of  making  decisions  concerning 
actions  to  be  taken  as  a  result  of  the  acquired  information. 

Communicating  Instructions  and  Orders.  The  process  of  transmit¬ 
ting  decisions  and  decision  related  orders  and  instructions  to  those 
parts  of  the  unit  that  must  implement  them. 

e  Monitoring.  The  process  of  obtaining  follow-up  information  about 
the  results  of  the  plans  and  orders. 

The  process  variables  provide  a  source  of  data  concerning  command  group 
behavior  which  should  be  related  to  the  performance  of  clusters  of  ARTEP 
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subtasks.  However,  it  was  not  known  to  what  extent,  if  any,  they  would 
be  related  to  performance  of  the  sub tasks  and,  thus,  they  constituted 
another  potentially  independent  or  nearly  independent  source  of  information 
concerning  the  battalion's  performance.  Although  data  are  reported  in 
subsequent  sections  on  the  process  measures,  time  constraints  precluded 
examining  the  relationships  among  subtasks  and  process  variables.  Such 
analyses  will  be  performed,  however,  and  included  in  subsequent  separate 
documentation. 

Objective  Measures 

ARTEP  Related.  Objective  performance  measures  were  developed  for  a 
subset  of  ARTEP  subtaaka  whose  standards  were  amenable  to  more  rigorous 
quantification.  These  more  objective  measures  were  developed  with  the 
aid  of  military  experts  and  through  a  process  which  involved  decomposing 
a  subtask  into  its  more  fundamental  elements.  For  example,  the  develop¬ 
ment  of  a  plan  for  the  resupply  of  units  in  contact,  one  of  the  S4  ARTEP 
subtasks,  consists  of  a  number  of  discrete  as  opposed  to  continuous  com¬ 
ponents,  e.g.,  planning  the  refueling  operation,  calculating  time-distance 
factors  between  supporting  end  supported  units  and  establishing  priority 
for  the  prepositioning  of  antiarmor  ammunition.  Once  the  S4  had  developed 
the  plan,  a  cognizant  member  of  the  controller  staff  could  merely  deter¬ 
mine  whether  or  not  each  element  was  represented.  Even  though  the  example 
provided  represents  a  dlcotomous  measure  in  the  most  fundamental  sense, 
selected  controllers  actually  responded  in  one  of  four  ways  to  the  presence 
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or  absence  of  target  behaviors.  These  four  responses  were:  "Yes,  the 
behavior  occurred;"  "Yes,  the  behavior  occurred  but  only  with  a  specific 
probe(s)  injected  to  elicit  it;"  "No,  the  behavior  did  not  occur;"  and 
"No,  the  behavior  did  not  occur  even  with  probing."  The  responses  were 
assigned  scores  of  4,  3,  2,  and  1  respectively,  reflecting  the  assumed 
ordinal  relation  of  the  alternatives.  Therefore,  no  assumptions  were 
made  as  to  the  equality  of  intervals  between  the  response  alternatives 
on  this  set  of  discrete  scales  and  since  probing  was  left  at  the  discre¬ 
tion  of  the  controllers,  it  was  not  possible  to  control  the  number  of 
probes  Injected  in  an  attempt  to  elicit  the  appropriate  behavior.  The 
scores  for  each  sub-element  or  component  of  the  various  subtasks  were 
averaged  to  produce  a  more  objective  measure  of  subtask  performance. 

Nineteen  of  the  47  ARTE?  sub tasks  identified  in  Appendix  B  could 
be  more  objectively  quantified.  These  subtasks  and  the  controllers  who 
were  responsible  for  their  evaluation  are  as  follows: 

Controller  Sub tasks 


Brigade  S2 
Brigade  S3 
Brigade  S1/S4 


IB,  2B,  2D,  3F,  5A,  and  5D 

1A,  1C,  ID,  IE,  3D,  6A,  6B,  and  8C 

3J  and  9D 


Fire  Support  1L,  7 A,  and  7B 

As  can  be  seen,  each  controller  responded  to  a  unique  set  of  measures 
devised  for  the  tasks  shown  above.  In  one  instance,  for  subtask  1L, 
the  subtask  was  broken  down  into  three  sub-subtasks,  each  of  which  was 
then  decomposed  into  more  fundamental  elements.  1L  sub-sub tasks  for 
which  scores  appear  later  in  the  report  are:  coordination  with  the  FO; 


informing  the  company  commanders  of  the  fire  plan;  and  informing  the 
supporting  artillery  of  the  fire  plan.  It  was  necessary  to  deviate 
from  the  four  category  discrete  response  format  for  three  subtasks 
included  in  this  category.  These  subtasks  are  IE,  3A,  and  3D.^  The 
measures  obtained  for  each  of  these  subtasks  are  as  follows: 

IE.  Select/control  key  terrain 

»  Number  of  key  terrain  features  in  the  area  of  operation  used 
by  the  battalion. 

•  The  total  number  of  key  terrain  features  in  the  battalion's 
area  of  operation. 

•  The  number  of  barriers,  obstacles,  and  reinforced  terrain 
used  by  the  battalion. 

3A.  Determine  critical  place 

»  The  grid  coordinates  of  the  location  the  command  group  identi¬ 
fied  as  being  the  enemy's  main  thrust  area. 

•  The  grid  coordinates  of  enemy  avenues  of  approach  identified 
by  the  command  group . 

3D.  Select  control  measures 

»  The  number  of  control  measures  used  at  each  of  the  following 
type  boundaries;  check  points,  coordinating  points,  contact 
points,  link-up  points. 

^ These  measures  could  have  easily  been  included  in  the  experimental 
category  because  they  are  more  speculative  than  the  other  measures  dis¬ 
cussed  here.  However,  since  they  were  developed  with  specific  subtasks 
in  mind,  they  were  included  here. 
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Single  measures  were  derived  from  Che  raw  numerical  data  provided 
for  each  of  chree  subcasks. 

EXPERIMENTAL  MEASURES 

The  measures  subsumed  under  this  rubric  represent  an  atcempc  to 
quantify  variables  which  prior  research  and  deductive  analyses  have 
shown  or  suggested  Impact  significantly  on  the  effectiveness  of  the  unit. 
Included,  in  this  category  as  well  are  speculative  measures  of  battle¬ 
field  outcomes,  measures  that  should  logically  be  related  to  the  pro¬ 
ficiency  of  the  command  group  as  reflected  in  those  conventional  param¬ 
eters  of  their  performance  discussed  above.  These  measures.,  as  opposed 
to  the  command  group  performances  Investigated,  do  not  command  a  concensus 
of  opinion  within  the  Army  community  vis-a-vis  whether  or  not  they  are 
important  or  are  critical.  Therefore,  one  can  only  conjecture  as  to  their 
suitability  for  performance  assessment.  Some  of  the  measures,  e.g.,  locus 
of  control,  are  speculative  enough  in  the  context  of  the  scenario  used  so 
as  to  make  it  Impossible  to  state  on  an  a  priori  basis  whether  more  or 
less  of  the  variable":  presence  is  "good"  or  appropriate.  Whether  it  is 
good  or  not  is  probably  greatly  dependent  upon  the  particular  scenario 
being  played  and  the  type  of  terrain  on  which  the  battle  is  fought.  How¬ 
ever,  for  ocher  measures,  e.g.,  the  number  of  times  the  task  force  units 
become  decisively  engaged  during  the  covering  force  operation,  it  was 
possible  to  deduce  what  should  represent  more  superior  performance,  i.e., 
the  less  frequently  the  units  become  decisively  engaged,  the  better  one 
could  judge  their  performance  to  be. 
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Because  of  the  nature  of  the  measures  falling  within  this  category, 
they  were  not  included  in  analyses  reported  dealing  with  the  training 
effectiveness  of  CAMMS.  For  the  training  effectiveness  analyses,  only 
th*  command  group  performance  measures  described  in  the  previous  section 
were  examined.  The  experimental  measures  were  only  entered  into  correla¬ 
tional  analysis  to  determine  if  performance  of  them  was  in  some  way  re¬ 
lated  to  the  more  conventional  measures  of  command  group  proficiency. 

As  was  the  case  for  command  group  performance  measures,  the  experi¬ 
mental  measures  can  be  more  or  less  clearly  divided  into  objective  and 
subjective  categories.  The  measures  falling  within  these  two  broad 
categories  and  the  method  used  to  derive  them  are  briefly  discussed 
below. 

Subjective  Measures 

Mission  Accomplishment  and  Components  Thereof.  The  chief  controller 
and  TOC  monitor  provided  estimates  as  to  the  extent  to  which  the  command 
group  was  able  to  accomplish  major  tasks  of  the  covering  force  mission, ^ 
the  mission  which  was  played  for  pre-  and  post-test  and  training  phases 
for  each  battalion.  Additionally,  a  global  measure  of  mission  accomplish¬ 
ment  was  obtained.  A  description  of  the  type  of  performance  measured  is 
the  following: 

•  Enemy  Thrust.  This  is  a  measure  of  the  degree  to  which  the 
battalion  command  group  was  able  to  identify  the  major  enemy  thrust. 

^ Department  of  the  Army  FM  71-2,  The  Tank  and  Mechanized  Infantry 
Battalion  Task  Force,  30  June  1977. 
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The  controllers  indicated  whether  the  major  thrust  had  been  identified 
within  1,  1-3,  3-5  and  5  or  more  kilometers.  For  purposes  of  data 
analysis,  these  categories  were  scored  1,  2,  3,  and  4  respectively  and, 
thus,  a  lower  score  should  indicate  more  superior  performance. 

•  Decisive  Engagement.  This  Is  simply  the  number  of  times  the  task, 
force  became  decisively  engaged.  The  ratings  provided  were  divided  into 
four  categories;  never,  once,  twice,  and  three  or  more  times,  which  were 
assigned  scores  ranging  from  1  through  4  respectively  for  analysis 
purposes . 

e  Task  Force  Losses.  This  is  an  estimate  of  the  friendly  losses 
which  were  incurred  during  the  operation.  One  of  five  categories 
of  loss  was  selected  by  the  raters.  These  categories  ranged  from  OZ  to 
50Z  In  10Z  increments  through  the  fourth  category  and  more  than  50Z 
losses  constituted  the  fifth  response  category.  This  scale  was  assigned 
numbers  1  through  5,  with  five  representing  the  greatest  loss  of  forces. 

e  OPFOR  Losses.  This  variable  is  the  counterpart  of  the  friendly 
force  loss  estimate.  Therefore,  the  same  rating  scale  was  used  by  the 
two  controller  raters  who  provided  this  estimate. 

e  Mission  Accomplishment.  This  constituted  an  overall  estimate  of 
the  extent  to  which  the  battalion  accomplished  its  assigned  mission. 

The  components  of  the  task  force  mission  described  above  should  represent 
components  of  this  overall  estimate.  A  five-point  rating  scale  was  used 
with  "not  at  all  effective"  and  "extremely  effective"  constituting  the 
anchors  at  the  low  and  high  ends  respectively. 
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These  measures  constituted  the  only  type  of  subjective  measures 
examined  within  this  more  speculative  category  of  variables.  They 
logically  should  be  related  to  more  conventional  staff  and  coamand  per¬ 
formance  parameters  when  examined  across  all  groups. 

Objective  Measures 

Locus  of  Control.  Lessons  learned  from  the  1967  and  1973  Mid-East 

t 

Wars  suggests  that  the  extent  to  which  control  is  centralized  can  sig¬ 
nificantly  affect  the  performance  of  fighting  units.  The  Israelis,  as 
opposed  to  their  enemies,  greatly  decentralized  control  of  forces  and 
other  assets  to  their  battalion  commanders,  which  made  it  possible  for 
them  to  capitalize  on  fleeting  moments  of  opportunity  on  the  battlefield. 
It  created  conditions  which  were  conducive  to  improvisation  and  Innovation 
that  would  have  otherwise  been  impossible.  Although  the  contribution  of 
this  variable  to  Israeli  successes  could  have  been  purely  a  function  of 
the  nature  of  the  requirements  being  faced  and  the  terrain  on  which  the 
battle  was  fought,  it  seemed  reasonable  to  investigate  its  influence 
within  the  context  of  the  present  investigation. 

To  measure  this  variable,  a  matrix  was  developed  with  organizational 
level  (echelon  within  the  battalion  to  which  control  could  be  vested) 
and  category  of  asset  (that  which  could  be  controlled)  forming  the  major 
axes.  This  matrix  is  shown  below. 
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Matrix  Used  to  Develop  the  Locus  of  Control  Measures 


Battalion 


The  OPORD  was  examined  by  one  member  of  the  research  team  in  order 
to  fill  in  the  matrix.  In  almost  all  instances,  the  organizational  unit 
having  control  of  given  assets  was  identified  in  the  Order.  In  those 
instances  where  it  was  not,  control  was  assumed  to  reside  at  the  organi¬ 
zational  level  to  vhich  it  normally  would  have  been  assigned.  The  number 
of  assets  controlled  by  each  organizational  level  was  multiplied  by  an 
arbitrary  weight  (1  through  5  for  squad  through  battalion  commander 
respectively)  and  these  products  were  averaged  across  all  command  levels 
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to  produce  a  final  score.  Thus,  the  higher  the  final  score,  the  more 
centralized  was  the  control  of  assets  within  the  battalion.  Although 
this  is  a  crude  approximation  of  reality,  it  should  be  somewhat  indica¬ 
tive  of  the  locus  of  control  within  the  unit. 

Intelligence  Operations  and  Reaction  to  Battlefield  Contingencies. 

4. 

Barber  and  Kaplan  (in  press)  in  previous  and  ongoing  work  with  another 
battlefield  simulation  undergoing  development  at  CAC,  the  Combined  Arms 
Tactical  Training  Simulator  (CATTS),  have  found  that  of  the  conventional 
ARTEP  command  group  performances  examined,  two  general  areas  appear  to 
be  of  especially  great  importance.  These  areas  are  intelligence  proces¬ 
sing®  and  the  ability  of  the  command  group  to  perform  all  these  activi¬ 
ties  implicit  in  rapidly  shifting  forces  in  response  to  battlefield 
contingencies  in  order  to  mass  the  forces  at  the  appropriate  time  and 
place.  These  general  areas  have  been  found  to  be  important  for  two 
reasons.  Relative  to  the  performance  of  other  ARTEP  related  measures, 
these  areas  are  the  ones  on  which  the  command  groups  perform  poorly  and, 
yet,  they  are  ones  that,  on  a  tentative  basis,  appear  to  capture  the 
largest  amount  of  variance  in  overall  command  group  performance  measures. 
Given  these  findings,  it  appeared  that  these  two  areas  should  be  the 
foci  of  attempts  to  develop  more  quantitative  measures  of  performance. 
Such  measures  might  not  be  practical  to  gather  for  feedback  purposes 

^It  was  assumed  that  this  process  consists  of  two  components: 
information  acquisition  and  transmission,  and  the  processing  of  raw 
information  into  intelligence.  The  measures  discussed  here  address 
primarily  the  information  acquisition  and  transmission  component. 
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during  a  unit's  normal  play  of  the  simulation,  but  they  could  prove  to 
be  useful  for  future  research  purposes  in  pursuit  of  the  overall  objec¬ 
tives  of  TEA  85.  Accordingly,  an  attempt  was  made  to  identify  quanti¬ 
tative  measures  which  (1)  judgmental  analysis  suggested  should  relate 
to  command  group  performance  and,  (2)  could  be  readily  obtained  during 
the  play  of  the  simulation.  A  number  of  such  measures  were  identified, 
the  preponderance  of  which  were  associated  with  the  shift  and  concen¬ 
trate  forces  performance  area.  Since  the  intelligence  area  and  opera¬ 
tions,  of  which  concentration  and  shifting  of  combat  forces  is  a  large 
part,  are  so  closely  related,  it  was  not  always  possible  to  clearly 
distinguish  between  the  two  In  terms  of  the  measures  developed.  Keeping 
this  fact  in  mind,  a  description  of  the  measures  grouped  into  the  intel¬ 
ligence  and  concentration  of  forces  areas  is  presented  below. 

e  Intelligence  Processing.  Two  measures  conjectured  to  be  related 
to  this  general  perfozmance  area  were  identified.  The  first  had  to  do 
with  communications  within  the  battalion.  Communications  transmissions 
examined  were  those  from  (1)  the  company  commanders  requesting  information 
from  the  battalion,  (2)  the  company  commanders  providing  information  to 
the  battalion,  (3)  the  battalion  to  the  company  commanders  providing 
information  and,  (4)  the  battalion  to  the  company  commanders  requesting 
information.  The  command  net  was  monitored  for  a  fifteen  minute  period 
during  each  hour  of  the  exercise.  Messages  transmitted  and  received 
were  categorized  into  those  predetermined  categories  identified  above. 
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The.  number  of  transmissions  of  e  ch  type  which  occurred  during  the  15- 
minute  interval  sampled  each  hour  of  the  pre-  and  post-test  and  training 
exercises  were  summed  and  averaged.  Thus,  the  average  number  of  transmis¬ 
sions  of  each  type  per  hour  by  exercise  were  available  for  analysis. 

The  second  measure  conjectured  to  be  related  to  the  intelligence 
process  was  the  ability  of  lower  level  task  force  units  to  ''see'*  the 
battlefield.  "Seeing"  the  battlefield  is  obviously  in  part  dependent 
upon  the  extent  to  which  that  battlefield  can  be  observed  from  assigned 
battle  positions.  Accordingly,  the  distance  between  each  task  force 
unit's  battlefield  position  and  man-made  or  natural  obstacle  or  terrain 
feature  which  obscured  line  of  sight  was  obtained.  Connecting  the  points 
(grid  quadrants)  of  the  obstacles  and  terrain  features  formed  a  polygon 
whose  area  could  be  readily  calculated.  These  calculations  were  made  for 
each  of  the  battle  positions  and  the  areas  were  summed  and  averaged.  Thus, 
the  score  produced  reflects  how  much  clear  viewing  area  was  available  to 
any  given  task  force  unit. 

•  Shift  and  Concentrate  Forces.  Five  measures  were  developed  which 
indirectly  could  reflect  the  ability  of  the  command  group  to  effectively 
perform  this  complex  set  of  activities.  These  measures  and  the  procedure 
used  to  derive  them  are  briefly  described  below. 

••  Distance  between  battle  positions  and  supply  points.  The  dis¬ 
tance  between  each  of  the  battle  positions  and  the  main  and  alternate 
supply  points  specified  in  the  OPORD  were  obtained  and  averaged  across 
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all  baccle  positions.  The  resultant  mean  distance  perhaps  should  reflect 
how  well  the  units  could  react  to  contingencies  on  the  battlefield  since 
anticipation  of  such  conditions  should  have  been  explicit  in  planning  the 
battle  and  supply  positions  established.  Thus,  to  a  certain  extent,  intel¬ 
ligence  preparation  before  the  battle  and  fidelity  of  logistical  (S1/S4) 
and  operations  (S3)  coordination  could  be  reflected  in  this  measure  as 
well. 

••  Distance  among  battle  positions.  The  distance  among  all  battle 
positions  as  specified  in  the  OPORD  and  established  on  the  game  board  was 
obtained  and  averaged  across  battle  positions.  The  mean  obtained  reflects 
the  average  distance  between  any  given  battle  position  and  all  other  bat¬ 
tle  positions.  Identification  of  the  correct  location  of  the  enemy’s  main 
thrust  should  determine  in  large  part  how  the  friendly  forces  are  initial¬ 
ly  positioned,  whether  in  tighter  mass  or  spread  more  loosely  across  the 
anticipated  battle  area(s) .  It  should  be  reflective  as  well  of  a  scheme 
of  maneuver  which  anticipates  that  rapid  shifts  of  combat  powev  may  be 
required  as  the  battle  progresses. 

#•  Number  of  battle  positions.  This  measure  should  be  highly 
related  to  the  one  above  and  co'.rld  reflect  anticipated  exigencies  that 
may  develop.  The  measure  was  straightforwardly  obtained  by  counting  the 
number  of  battle  positions  psecified  in  the  OPORD. 

••  Distance  of  specified  battle  positions  from  each  battle. 

Again,  this  measure  was  conjectured  to  reflect  the  ability  to  anticipate 
contingencies  and  develop  compromises  in  terms  of  positioning  of  forces 
that  takes  into  account  the  knowns  and  unknowns  of  the  situation.  To 
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calculate  this  measure,  the  distance  between  assigned  battle  positions 
for  the  various  task  force  elements  and  the  point  at  which  the  battle 
actually  took  place  was  obtained.  These  distances  were  then  averaged 
for  all  task  force  elements. 

••  Engagement  range  of  all  battles  fought.  This  measure  is 
probably  a  more  Indirect  than  a  direct  measure  of  command  group  perfor¬ 
mance.  It  should  reflect  the  ability  of  the  task  force  units  to  success¬ 
fully  Implement  the  covering  force  mission  as  the  parameters  of  it  have 
been  specified  in  the  OPORD.  If  one  assumes  that  such  a  mission  carries 
with  It  the  task  of  engaging  the  enemy  at  the  maximum  range  possible,  to 
attrlt  their  forces  to  the  greatest  extent  possible,  while  at  the  same 
time  to  minimize  losses  and  expenditures  of  resources  of  the  friendly 
forces,  then  the  greater  the  range  at  which  skirmishes  occur  within  a 
given  envelope  should  be  Indicative  of  relatively  superior  performance. 

A  number  of  parameters  are  therefore  likely  to  contribute  to  how  this 
particular  measure  might  behave  and,  .in  that  sense,  it  is  even  more 
speculative  than  the  others  included  in  the  general  experimental  variable 
category.  This  measure  was  obtained  at  that  point  in  time  when  the 
friendly  and  OPFOR  controllers  mutually  agreed  that  an  engagement  should 
taice  place.  The  distance  between  the  opposing  forces  was  obtained  for 
each  engagement  of  each  task  force  unit  and  averaged  across  all  engage¬ 
ments.  Thus,  the  score  produced  represents  the  average  distance  between 
opposing  forces  for  any  given  engagement. 
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Battlefield  outcomes.  Measures  of  this  variety,  if  they  can  be 


produced  with  any  degree  of  fidelity  via  simulation,  should  not  neces¬ 
sarily  for  any  given  unit  but  should  across  a  number  of  units  bear  some 
relationship  to  the  performance  capabilities  of  the  command  and  control 
process.  To  explore  this  relationship,  data  were  gathered  on  three 
crude  battlefield  outcome  measurer.  These  were  number  of  friendly 
forces  lost,  number  of  friendly  force  weapons  lost,  and  amount  of  ground 

Q 

lost7  during  the  operation.  The  first  two  measures  were  derived  from 
summaries  of  friendly  unit  weapons  and  personnel  status  summaries  that 
were  provided  on  an  hourly  basis  by  the  computer.  The  losses  calculated 
during  each  hour  were  summed  over  hours  and  averaged  to  produce  the  mean 
number  of  losses  that  occurred  during  any  given  hour.  No  differentiation 
as  to  type  of  weapons  system  or  category  of  personnel  was  made  in  the 
calculations.  Further,  although  the  exercise  SOP  specified  maximum 
limits  for  resupply  action  for  given  time  periods,  it  was  not  possible 
to  ascertain  how  rigidly  the  units  adhered  to  these  specifications.  The 
amount  of  ground  lost  was  obtained  by  merely  measuring  the  distance 
between  the  front  line  traces  at  the  beginning  and  end  of  each  exercise. 

All  the  various  measures  within  this  experimental  category  were 
gathered  during  the  pretest,  the  training  and  the  post  test  exercises. 

^In  a  covering  force  mission,  it  is  recognized  that  losing  a  speci¬ 
fied  or  implied  amount  of  ground  within  a  specified  amount  of  time  is 
expected.  However,  given  that  the  amount  of  ground  to  be  lost  was  a 
constant,  losing  more  or  less  than  one  should  have  is  indicative  of  how 
well  the  unit  performed. 
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However,  for  purposes  of  the  correlational  analyses  presented  in  the  next 
section,  only  the  training  exercise  data  were  used  because  a  more  repre¬ 
sentative  sample  of  the  behaviors  being  measured  could  be  obtained  during 
this  extended  session. 

Reliability  of  Subjective  Measures 

As  discussed  implicitly  above,  ratings  were  provided  by  multiple  con¬ 
trollers  for  almost  all  of  the  subjective  command  group  and  experimental 
measures.  The  TOC  monitor  provided  a  comprehensive  set  of  evaluations 
for  the  measures  in  question.  The  Brigade  S2  and  S3,  the  FSO,  and  chief 
controllers  each  rated  subsets  of  those  on  which  the  TOC  monitor  provided 
evaluations.  Thus,  the  paired  ratings  were  comprised  of  the  TOC  monitor's 
evaluations  in  combination  with  one  of  the  lattermentioned  controllers 
for  most  measures.  To  obtain  an  estimate  oi  the  amount  of  agreement 
across  the  two  sets  of  common  ratings,  they  were  intercorrelated  sepa¬ 
rately  for  the  battalions  drawn  from  each  of  the  two  participating  divi¬ 
sions.  Also,  the  data  within  a  division  was  examined  across  all  bat¬ 
talions  and  all  exercises  (pre-,  training,  and  post)  in  which  those  units 
played.  The  results  of  this  analysis  are  shown  in  Table  1.  The  correla¬ 
tions,  with  only  three  exceptions,  are  all  statistically  significant. 
However,  in  terms  of  their  practical  significance,  there  is  not  strong 
agreement  among  the  raters,  particularly  the  TOC  monitor  and  the  FSO. 

10It  was  necessary  to  examine  interrater  reliability  by  division  since 
the  personnel,  including  the  TOC  monitor,  who  provided  the  data  were  dif¬ 
ferent  between  the  two  divisions.  This  lack  of  uniformity  in  controller/ 
rater  staff  is  more  fully  discussed  in  the  next  section. 
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TABLE  1 

Interrater  Agreement  for  Command 
Group  Performance  and  Experimental  Measures 

Division  //I  Division  #2 
Type  of  Measure  (3  Battalions)  (2  Battalions) 

•  ARTEP  Subtasks 


••  TOC  Monitor  and  S2 

.33* 

.42* 

(n  -  92)1 

(n  -  59) 

ee  TOC  Monitor  and  S3 

.19* 

.28* 

(n  -  142) 

(n  -  103) 

•o  TOC  Monitor  and  FSO 

.16 

-.08 

(n  -  45) 

(n  -  40) 

• 

Organizational  Processes 

••  TOC  Monitor  and  Chief  Controller 

.37* 

-.02 

(n  -  53) 

(n  ■  30) 

• 

Overall  Performance  Measures 

•e,  TOC  Monitor  and  Chief  Controller 

.39* 

.62* 

(n  -  49 

(n  -  35) 

• 

Mission  Accomplishment 

ee  TOC  Monitor  and  Chief  Controller 

.61* 

.43* 

(n  -  49) 

(n  -  35) 

♦Significant  at  .01  level  or  beyond 

^The  n's  shown  are  the  number  of  subtasks  or  items  common  for  both 
raters  during  all  three  exercises  across  all  participating  battalions. 


The  average  correlation  across  both  divisions  and  all  measures  is  .34 
indicating  on  balance  that  only  slightly  less  that  12Z  of  the  variance 
in  the  two  sets  of  ratings  is  shared  in  common.  This  suggests  the 
evaluations  provided  by  the  raters  tended  to  be  unique  rather  than 
common,  a  situation  which  may  have  resulted  from  (1)  the  insensitivity 
of  the  instruments  used,  (2)  unique  samples  of  the  same  behaviors  being 
observed  from  distinct  vantage  points,  (3)  restriction  of  range  in  most 
of  the  ratings  which  decreases  the  likelihood  of  detecting  the  extent 
of  rater  reliability  via  conventional  correlational  techniques  and 
(4)  some  combination  of  these  or  other  factors.  The  question  as  to 
whether  or  not  to  combine  the  subjective  evaluations  of  the  various 
raters  would  have  been  rhetorical  or  nearly  so  if  interrater  reliability 
had  been  found  to  be  relatively  high  (e.g.,  .70  or  above).  This  ques¬ 
tion  obviously  became  one  of  real  concern  when  the  reliabilities  were 
found  to  be  moderately  low.  A  decision  was  made  to  combine  the  ratings, 
as  implicitly  indicated  in  previous  portions  of  this  measurement  sec¬ 
tion.  The  decision  was  predicated  upon  the  assumption  that  aggregation 
of  the  ratings  would  perhaps  provide  a  better  measure  of  reality  than 
either  of  the  unique  aspects  of  it  that  the  raters  were  presumed  to  have 
observed.  Additionally,  had  the  assumption  tl.et  the  raters  differed 
because  they  were  responding  to  unique  aspects  of  the  same  behavior  been 
fallacious,  one  would  still  have  been  thrust  into  a  quandry  as  to  which 
of  the  two  ratings  was  the  most  veridical,  a  question  that  could  not 
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have  been  addressed  with  the  data  collected  in  this  preliminary  investi¬ 
gation.  The  problem  of  low  rater  interreliability  and  the  potential 
positive  or  negative  impact  of  combining  the  ratings  are  additional  con¬ 
siderations  that  must  be  taken  to  account  with  the  other  limiting  factors 
recountered  herein  in  interpreting  the  results  of  this  effort. 

PROCEDURE 

Data  Collection  Team 

A  team,  consisting  of  members  provided  by  the  (1)  division  of  the 
participating  units  and  (2)  CAC  and  ARI,  collected  data  on  all  command 
group  performance  and  experimental  measures.  The  brigade  level  control¬ 
lers  and  game  board  players  were  provided  by  the  participating  unit's 
parent  organization.  The  same  individuals  served  in  their  respectively 
assigned  roles,  ones  for  which  they  had  prior  experience  in  all  but  a 
few  isolated  Instances,  for  all  exercises  conducted  within  a  particular 
division. 

The  TOC  monitor  was  a  lieutenant  colonel  provided  from  an  element  of 
CAC  under  whose  auspices  and  direction  the  battalion  command  group  ARTEP 
was  developed.  Thus,  he  was  very  familiar  with  staff  procedure  and 
operations  at  the  battalion  level.  Although  it  was  initially  planned 
for  the  same  individual  to  accompany  the  collection  team  to  both  par¬ 
ticipating  divisions,  for  varying  reasons  this  was  not  possible.  The 
alternate  TOC  monitor,  who  served  as  a  data  collector  during  those  exer¬ 
cises  conducted  at  the  second  division,  however,  was  assigned  to  the 
same  element  of  CAC  from  which  the  first  monitor  came.  Even  though 


these  two  Individuals  spent  some  time  together  in  an  attempt  to  form  a 
unified  frame  of  reference  from  which  their  ratings  would  be  made,  it  is 
not  possible  to  state  definitively  that  this  objective  was  achieved.  This 
condition  then  obviated  the  possibility  of  examining  the  extent  of  agree¬ 
ment  between  the  TOC  monitor  and  the  chief  controller  (who  was  the  same 
individual  across  all  exercises)  for  those  instances  where  they  provided 
comparable  measures.  Furthermore,  initial  plans  called  for  a  second 
observer  who  would  have  performed  the  TOC  monitor  functions  in  the  combat 
trains  area;  a  condition,  which  if  fulfilled,  would  have  provided  a  con¬ 
sistent  frame  of  reference  for  the  S1/S4  functional  areas.  Again,  this 
requirement  could  not  be  fulfilled  and,  thus.  It  was  impossible  to  collect 
data  totally  consistent  across  all  units  in  the  S1/S4  performance  area. 

The  chief  controller,  one  of  the  primary  subjective  data  providers, 
also  came  from  CAC  and  was  assigned  to  the  element  responsible  for 
developing  CAMMS.  Thus,  he  was  very  familiar  with  the  simulation  and 
had  served  as  chief  controller  for  at  least  10  battalion  level  CAMMS 
exercises  before  participating  in  this  effort.  The  data  which  he  was 
responsible  for  providing  and  the  data  collection  requirements  of  the 
TOC  monitor  and  those  brigade  level  controllers  provided  by  the  unit 
were  specified  in  the  previous  section. 

Two  ARI  personnel  formed  the  final  component  of  the  data  collection 
team.  These  individuals  gathered  the  raw  data  from  which  the  objective 
experimental  measures  were  derived.  They  also  monitored  ongoing  activi¬ 
ties  during  the  conduct  of  the  exercise  to  insure  that  procedure  was 
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being  followed  vis-a-vis  experimental  design  and  control  measures  and 
assisted  any  of  the  data  collectors/raters  who  had  difficulty  responding 
to  or  understanding  the  data  collection  instruments. 

Test  and  Training  Exercises 

The  procedure  followed  to  gather  data  related  to  the  subjective  and 
objective  command  group  performance  measures  and  the  more  speculative 
experimental  measures  is  shown  in  Figure  2.  The  entire  sequence  of 
events,  from  orientation  session  through  the  post-test  debriefing  session, 
took  place  over  a  four-day  period  with  no  more  than  ten  hours  of  game 
play  occurring  on  any  given  day  for  the  three  exercises.  Prior  to  initi¬ 
ating  the  first  exercise,  an  orientation  session  was  conducted  for  the 
players  and  controllers.  During  the  first  part  of  this  session,  both 
the  controllers  and  players  were  present.  They  were  briefed  by  ARI  per¬ 
sonnel  and  the  chief  controller  concerning  the  overall  objectives  of  the 
ARTS  effort  and  what  part  the  present  investigation  and  they  would  play 
in  accomplishing  those  objectives.  Further,  they  were  familiarized  with 
the  experimental  design  and  the  schedule  of  events  which  that  design 
would  entail  as  well  as  the  general  scenario  chat  would  be  utilized  in 
each  of  the  exercises.  Special  requirements  and  constraints  that  would 
be  required  were  discussed  and  the  participants  were  told  that  a  report 
summarizing  the  outcome  of  the  exercises,  but  which  would  not  Identify 
the  battalions  in  terms  of  their  respective  performances,  would  be  pro¬ 
vided  to  the  division  comnander.  The  command  group  was  provided  copies 
of  the  training  objectives  for  the  exercises  which  were,  in  effect. 


Pre-Test  Exercise  Training  Exercise  Post-Test  Exercise 
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the  47  ARTEP  subcasks  on  which  most  of  the  data  were  to  be  collected. 
During  the  second  part  of  the  session,  only  the  controllers  were  present. 
They  were  (1)  introduced  to  the  data  collection  forms  and  provided  in¬ 
struction  on  how  to  use  them,  (2)  provided  guidance  concerning  more  spe¬ 
cialized  procedural  requirements  than  those  which  were  covered  in  the 
general  session,  and  (3)  given  training  concerning  how  to  perform  the 
various  roles  and  functions  associated  with  the  mechanics  of  the  game 
as  these  factors  related  to  the  controllers'  respective  positions.  A 
question  and  answer  period  was  held  at  Che  end  of  the  controller  training 
session  in  order  to  verify  that  they  fully  understood  the  data  collection 
requirements  which  they  had  been  requested  to  fulfill  and  the  require¬ 
ments  of  the  game  which  their  assigned  roles  would  entail. 

Following  the  orientation  session,  the  pre-test  exercise  was  begun. 
During  this  time  the  chief  and  brigade  level  controllers  issued  the  bri¬ 
gade  OPORD  to  the  battalion.  The  battalion  command  group  spent  three 
hours  analyzing  the  order  and  developing  their  plans,  a  process  which 
culminated  in  issuing  their  OPORD  to  the  companies.  The  execution  phase 
of  the  exercise  began  with  the  issuance  of  the  battalion  OPORD. 

Following  the  execution,  the  data  which  had  been  collected  through¬ 
out  the  planning  and  execution  phases  were  consolidated  and  a  feedback 
session  was  held.  The  general  format  of  this  session  and  the  activities 
which  immediately  prefaced  it  are  shown  in  Figure  3.  A  general  observa¬ 
tions  briefing  was  held  first  with  all  members  of  the  command  group 
present.  The  chief  and  OPFOR  controllers  conducted  this  session  during 
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Figure  3.  Feedback  Session  Format.* 


which  chey  provided  feedback  based  upon  Cheir  observations  and  observa¬ 
tions  of  others,  as  well  as  some  of  the  quantitative  data  which  had  been 
collected.  Following  this  briefing,  each  member  of  the  command  group 
and  their  counterparts  on  the  controller  staff  met  in  separate  closed 
sessions.  During  these  sessions,  detailed  feedback  was  provided  as  to 
how  well  the  staff  member  had  performed  those  ARTEP  sub  tasks  which  fell 
within  his  area  of  responsibility.  The  feedback  was  limited  to  the  sub¬ 
jectively  derived  ARTEF  subtask  measures,  the  training  objectives  for 
the  exercise.  Specific  examples  of  incorrect  or  inappropriate  performance 
were  provided  for  each  sub  task  area  to  the  extent  this  was  possible.  Fol¬ 
lowing  the  one-on-one  feedback  sessions,  the  command  group  members  met 
collectively  In  closed  session  during  which  each  staff  member  hod  on 
opportunity  to  dlscuas  the  outcomes  of  the  one-on-one  session  and  propose 
to  the  battalion  commander  ways  of  overcoming  deficiencies  that  had  been 
identified.  Remedial  courses  of  action  were  developed  and  approved  fer 
Implementation  during  the  next  exercise. 

The  sequence  of  events,  from  the  issuance  of  the  brigade  OPORD 
through  the  feedback  session,  was  followed  for  the  training  and  post¬ 
test  exercises  with  one  exception.  For  the  latter  exercise,  only  a 
final  general  debriefing,  rather  than  a  full-blown  feedback  session, 
was  conducted.  In  terms  of  duration,  the  execution  phase  was  considerably 
lengthier  In  the  training  os  opposed  to  the  pre-  and  post-test  exercises 
for  obvious  reasons.  The  exact  duration  of  these  phases  is  shown  In 
Figure  2. 
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RESULTS 


TRAINING  EFFECTIVENESS 

Tho  primary  basis  used  for  assessing  training  effectiveness  was  the 
nature  and  amount  of  performance  change  occurring  subsequent  to  the 
diagnostic  (pre-test)  exercise.  Such  data  were  tabulated  for  all  appro¬ 
priate  dependent  measures  and  repeated  measures  analysis  of  variance 
(ANOVAs)  performed  to  determine  whether  any  of  the  differences  achieved 
statistical  significance.  For  those  performance  dimensions  that  achieved 
statistical  significance,  a  posteriori  least  significant  difference  tests 
were  performed  to  identify  which  of  the  differences  among  means  were  sig¬ 
nificant,  i.e.,  prs-test  vs.  post-test,  pre-test  vs.  training  and  training 
vs.  post-test.  Results  are  presented  and  discussed  for  each  major  tvpe 
of  performance  measure.  Change  in  performances  over  time  is  not  the  only 
ingredient  of  training  effectiveness  of  interest.  Ability  of  measures 
to  differentiate  or  distinguish  strengths  and  weaknesses  among  the  com¬ 
ponents  of  performance  is  important.  Accordingly,  the  data  were  also 
interpreted  from  this  perspective  for  each  major  type  of  performance 
measure.  Because  of  time  constraints  and  the  large  volume  of  pairwise 
comparisons  which  would  be  required  to  determine  nd  present  the  statisti¬ 
cal  significance  of  the  differences  among  th»*  ARTEP  tasks  and  subtasks, 
the  degree  to  which  these  measures  discriminated  performance,  as  reported 
subsequently,  is  based  on  only  visual  inspection  of  the  data. 
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Subjective  ARTEP  Subtask  and  Task  Ratings 


All  main  effects  and  interactions  for  both  the  exercise  by  subtask 
and  exercise  by  task  ANOVAs  were  statistically  significant  at  the  P<.001 
level  except  for  subtask  effects  which  were  significant  at  only  the  P<.05 
level.  Summary  ANOVA  tables  are  shown  in  Appendix  C.  Thus,  there  were 
differences  in  rated  performance  attributable  to  (1)  the  exercise  session, 
(2)  the  subtask  being  judged  and  (3)  the  tasks  comprised  of  those  sub¬ 
tasks;  and,  of  course,  the  interaction  indicates  that  differences  among 
exercise  sessions  were  not  uniform  across  all  subtasks  and  tasks.  The 
data  showing  the  change  in  mean  performance  across  the  three  exercise 
sessions  for  each  subtaak  are  shown  in  Table  2.  A  numerical  improvement 
in  mean  performance  between  the  first  exercise  session  (pre-test)  and 
the  third  (post-test)  was  obtained  for  46  of  the  47  3ubtasks.  Though 
only  13  of  these  differences  achieved  statistical  significance  at  P<.05 
level,  the  consistency  in  the  direction  of  the  differences  strongly  sug¬ 
gests  that  the  failure  to  achieve  a  far  greater  number  of  significant 
differences  is  more  likely  a  function  of  the  small  number  of  units  than 
it  is  the  absence  of  real  differences.  These  same  data  when  aggregated 
by  task  produce  findings  which  are  consistent  with  those  for  subtasks  as 
seen  in  Table  3  where  differences  between  pre-test  and  post-test  sessions 
are  significant  for  only  3  of  the  12  tasks  but  a  numerical  improvement  in 
mean  performance  was  obtained  for  all  12  tasks. 

Examination  of  the  differences  in  performance  between  the  first  (pre¬ 
test)  and  the  second  (training)  session  reveal  patterns  and  magnitudes 
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Significant  at  the  ,05  level  or  higher. 


very  much  like  those  of  the  differences  between  the  first  and  third 
session.  Eleven  of  the  47  subtasks  and  4  of  the  12  tasks  reached  the 
P<.05  level  of  significance.  A  numerical  improvement  in  mean  perfor¬ 
mance  occurred  for  41  of  the  subtasks  and  11  of  the  12  tasks. 

The  parallelism  of  findings  thus  far  noted  do  not  extend  to  differ¬ 
ences  in  performance  between  the  second  (training)  and  third  (post-test) 
sessions.  Again,  from  Tables  2  and  3  it  can  be  seen  that  differences 
were  significant  for  only  five  of  the  subtasks  though  three  tasks  still 
reached  the  P<.05  level.  Perhaps  more  revealing  is  the  generally  much 
smaller  differences  obtained  and  Che  fact  no  numerical  improvement  was 
obtained  for  19  of  the  subtasks  and  six  of  the  tasks.  This  apparent 
slowdown  in  the  learning  rate  is  suggestive  of  the  common  learning  curve 
asymptote.  This  will  be  discussed  along  with  other  plausible  explana¬ 
tions  in  a  subsequent  section  of  the  report  dealing  with  CAMMS  as  a 
training  research  vehicle. 

In  terms  of  differentiating  among  ARTEF  subtasks,  inspection  of 
the  subtask  means  within  exercise  session  as  shown  in  Table  2  reveals 
scores  ranging  from  a  low  of  1.30  for  one  subtask  to  a  high  of  4.20  for 
three  subtasks  in  the  pre-test.  Thirty-five  of  the  subtasks  received 
mean  ratings  below  4.00  which,  based  on  the  rating  scale  anchors  used, 
indicate  failure  to  meet  the  standard  of  performance.  On  the  other 
hand,  post- test  session  results  show  all  but  four  subtasks  performed  at 
or  above  the  standard.  The  range  in  means  for  that  session  extend  from 
3.10  to  a  high  of  4.60.  For  diagnostic  purposes,  the  data  can  be  viewed 
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in  terms  of  planning  functions  (Tanks  1  through  4)  and  execution  func¬ 
tions  (Tasks  5  through  12).  From  that  perspective,  the  subtasks  of 
identify  critical  combat  information  and  intelligence  (IB,  2A) ,  gather 
critical  information  and  intelligence  (2B) ,  analyze  opposing  force  (2C) , 
and  isseninate  critical  combat  information  and  Intelligence  (2D) , 
develop  a  communications  plan  (3F)  and  plan/ employ  active/passive 
security  measures  (31)  were  all  subtasks  observed  during  the  planning 
stages  that  were  rated  relatively  low  (i.e.,  mean  less  than  4.0).  Driving 
the  execution  phase,  a  similar  pattern  developed,  i.e.,  subtasks  dealing 
with  the  same  general  behaviors  were  rated  relatively  low.  These  low 
rated  subtasks  included  gather  critical  information  and  intelligence  (5B) , 
analyze  opposing  force  (5C),  disseminate  critical  combat  Information  and 
intelligence  (3D),  defeat  or  suppress  opposing  force's  electromagnetic 
Intelligence  effort  (10A) ,  and  react  t(  opposing  force  electronic  warfare 
(12A) . 

During  the  planning  stages,  those  subtask"  associated  with  the  fire 
support  area  were  also  generally  rated  relatively  lower.  However,  the 
fire  support  related  subtaaks  during  the  execution  phase  did  not  follow 
this  pattern. 

During  the  execution  phase,  subtasks  reined  to  the  admin/log  area 
such  as  arm  and  fuel  the  systems  (9A) ,  fix  the  system  (9B),  and  support 
the  troops  (9C),  were  relatively  low  rated.  However,  admin/log  related 
subtasks  were  not  rated  particularly  low  during  the  planning  phase.  These 
results  for  the  admin/log  are?  may  be  less  reliable  chan  most  of  the 
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others  because  of  the  limited  opportunity  of  the  admin/log  controllers, 
who  were  the  only  raters,  to  observe  the  performance  within  this  par¬ 
ticular  area. 

At  the  task  level,  and  using  the  same  criterion  as  in  the  subtasks, 
the  results  of  comparing  the  mean  ratings  are  basically  the  same  as 
obtained  by  looking  at  the  specific  subtasks,  but  at  a  grosser  level. 

The  results  of  the  analysis  of  the  performance  of  the  battalion  com¬ 
mand  groups  on  ARTE?  tasks  and  subtasks  indicate  that  three  major  areas 
of  command  group  performance  were  the  most  problematic  or  poorly  per¬ 
formed.  These  were:  intelligence,  fire  support,  and  admin/ log.  These 
findings  closely  parallel  the  informal  observations  of  the  CAMMS  Research 
Team.  They  also  correspond  to  previous  research  on  battalion  command 
group  ARTEP  performance  previously  examined  in  CATTS  exercises  (Barber 
and  Kaplan,  in  press) . 

Organizational  Processes 

Performance,  as  measured  by  ratings  from  1  -  not  effective,  to 
5  -  extremely  effective,  of  the  organizational  process  dimensions,  was 
analyzed  to  determine  whether  there  were  changes  occurring  across  ses¬ 
sions.  The  summary  ANOVA  table  in  Appendix  D  shows  no  statistically 
significant  effects  for  exercise  sessions,  process  dimensions  or  their 
interaction.  Examination  of  the  means  and  differences  in  Table  4  con¬ 
firms  that  even  on  a  nonstatlstical  numerical  basis  alone,  the  change 
across  sessions  is  very  slight  albeit  in  the  direction  of  improvement 
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from  pre-  to  post-test  and  from  pre-test  to  training.  In  terms  of  dif¬ 
ferentiation  for  diagnostic  purposes,  these  data  suggest  that  on  the 
average,  all  five  of  these  processes  are  performed  a  bit  better  than 
"somewhat  effective"  and  that  there  is  little  basis  for  distinguishing 
among  them.  Whether  this  la  attributable  to  the  small  sample,  the 
Insensitivity  of  the  measures  as  used  In  this  effort  or  other  factors 
cannot  be  determined  from  these  data,  Further  Implications  of  these 
data  will  be  discussed  In  subsequent  section  on  measurement. 

Overall  Performance 

The  ANOVA  for  overall  performance  ratings  obtained  for  each  staff 
element,  for  the  battalion  commander  and  for  the  collective  command 
group  are  contained  In  Appendix  E,  Exercise  effects  and  overall  com¬ 
mand  staff  effects  were  statistically  significant  at  the  ?<.001  level. 
When  these  data  were  analyzed  in  terms  of  simple  effects,  the  only 
statistically  significant  differences  among  exercise  sessions  were  for 
the  personnel,  logistics,  and  fire  support  functions.  As  shown  in 
Table  5,  all  three  differences  were  significant  for  personnel  but  only 
the  difference  between  pre-  and  post-tests  was  significant  for  logistics 
and  fire  support.  Like  the  ARTEP  ratings,  the  differences  in  mean 
numerical  values  between  pre-test  and  post-test  for  the  six  command 
staff  elements  Individually  and  collectively,  and  for  the  battalion 
commander  reflected  improved  performance  as  did  the  differences  between 
the  pre-test  and  training  exercise.  Again,  a  tendency  toward  asymptote 
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Significant  at  the  .05  level  or  higher. 


for  Che  change  in  performance  ratings  between  the  training  and  post-test 
exercises  seems  indicated. 

Examining  means  by  exercise  across  staff  functions  give  evidence  of 
differentiation.  Scores  range  from  a  low  of  1.20  to  a  high  of  4.10  in 
the  pre-test  session.  While  the  average  rating  for  all  but  three  of 
the  functions  is  above  the  "very  effective"  level  for  the  post-ter<t, 
there  is  still  a  range  of  2.80  to  4.50.  Again,  the  personnel  and 
logistics  areas  were  rated  relatively  lower  than  the  others. 

ARTEP  Related  Objective  Measures 

The  ANOVA  tables  for  the  five  staff  element/function  categories 
for  which  objective  measures  for  one  or  more  subtasks  were  obtained 
are  contained  in  Appendix  F.  In  terms  of  training  effectiveness  as 
measured  by  differences  among  the  three  exercise  sessions,  only  the 
exercise  effects  for  the  S1/S4  (admin/log)  element  achieved  statistical 
significance  at  the  P<.05  level.  Table  6  shows  that  within  this  func¬ 
tion,  only  one  of  the  two  subtasks  produced  significant  differences.  In 
this  case,  both  the  pre-test  -  poet-test  difference  and  the  training  - 
post- test  difference  achieved  significance  at  che  P<.05  level  and  both 
differences  were  in  the  direction  of  improved  performance. 

The  degree  to  which  the  objectively  rated  component  tasks  of  the 
various  staff  element  functions  produced  differential  performance  scores 
is  not  appropriately  addressed  for  these  measures  because  they  were  not 
designed  to  be  either  representative  or  inclusive  in  number  or  content 
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of  Che  respective  staff  element  functions.  However,  for  purposes  of 
completeness,  the  tables  of  means,  S.D's,  and  differences  for  the  S2, 

S3,  and  FSO  elements  are  Included  In  Appendix  G.  How  they  relate  to 
the  other  ratings  of  the  ARTEP  subtasks  from  which  they  were  derived 
will  be  treated  in  the  section  on  measurement. 

REFINEMENT  OF  PERFORMANCE  MEASUREMENT  PROCEDURES 

The  development  of  valid,  reliable  and  sensitive  measurement  tech¬ 
niques  to  determine  the  proficiency  level  of  battalion  command  groups 
was  one  of  the  objectives  of  this  effort.  As  pointed  out  In  the  methods 
section,  the  measures  used  can  be  roughly  dichotomized  into  those  that 
reflect  dimensions  of  performance  that  have  been  previously  used  and/or 
through  concensus  within  the  Army  community,  are  generally  accepted  as 
being  criteria  for  command  group  proficiency,  e.g.,  ARTEP  aubtasks,  and 
Into  those  which  are  much  more  speculative  in  nAture.  The  latter,  though 
having  a  rationale  and  ostensibly  a  more  objective  dimensionality,  are 
just  being  explored  in  regard  to  their  relationship  to  proficiency.  For 
these  more  speculative  measures,  the  concern  is  whether  and  how  these 
dimensions  relate  to  more  commonly  accepted  measures  of  command  group 
performance  and  what  difficulties  there  may  be  in  their  application  and 
interpretation.  Primary  attention  in  this  effort  was  on  the  latter  type 
measures  with  the  former  serving  as  quasi  criteria  but  both  being  examined 
in  terms  of  relation  to  various  measures  of  battlefield  outcomes. 

In  the  following  paragraphs  the  interrelations  among  the  various 
subjective  and  objective  measures  of  battlefield  outcomes  are  first 
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discussed.  Next,  the  relationships  of  the  battlefield  outcome  measures 
to  both  the  subjective  and  experimental  objective  measures  of  command 
group  performance  are  presented.  Finally,  the  relation  of  the  experi¬ 
mental  objective  measures  to  the  subjectively  assessed  ARTEP  subtasks 
is  discussed. 

Battlefield  Outcomes 

Table  7  shows  the  intercorrelations  among  the  outcome  measures.  Five 
of  the  correlations  are  statistically  significant  and  several  others 
approach  significance.  The  computer  generated  personnel  and  equipment 
losses  correlate  highly  with  each  other  suggeoting  that  the  algorithms 
presume  attrition  on  these  two  dimensions  to  be  closely  linked.  These 
same  two  dimensions  plus  area  lost  correlate  highly  and  negatively  with 
the  factor  enemy  thrust,  an  estimate  of  the  magnitude  of  error  in  identi¬ 
fying  where  the  major  penetration  will  occur.  This  suggests  that  friendly 
personnel,  equipment  and  area  losses  will  decrease  as  the  error  in  esti¬ 
mating  where  the  main  thrust  will  occur  increases.  This  paradoxical 
result  may  not  be  unreasonable  for  a  covering  force  operation.  It  could 
mean  that  sufficient  error  in  locating  the  enemy  thrust  v  11  reduce  the 
frequency  and  Intensity  of  fire  exchanges  and  thereby  decrease  personnel, 
equipment  and  ares  losses. 

The  substantial  negative  correlation  between  area  lost  and  number  of 
decisive  engagements  also  seems  plausible  in  that  more  frequent  decisive 
engagements  might  for  the  short  term  reduce  the  area  given  up  but  arc 
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TABLE  7 

Intercorrelations  Matrix  of  the  Relationships  Among 
the  Various  Battlefield  Outcome  Measures9 
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‘Significant  at  .05  level,  2  tailed  test 
‘Significant  at  .01  level,  2  tailed  test 


not  likely  to  result  in  better  mission  accomplishment.  The  absence  of  a 
correlation  between  number  of  decisive  engagements  and  mission  accom¬ 
plishment  tends  to  confirm  this  reasoning. 

It  is  reassuring  to  note  that  increases  in  the  rated  OPFOR  losses 
correlate  with  Improved  missions  accomplishment  scores.  Also,  success 
in  attrlting  the  enemy  is  positively  correlated  with  friendly  force 
losses,  a  finding  which  is  reasonable  and  to  be  expected. 

Subjective  Ratings  and  Battlefield  Outcome  Measures 

Because  of  the  experimental  nature  of  the  battlefield  outcome  meas¬ 
ures,  Insights  regarding  their  meaning  for  command  group  proficiency  were 
sought  by  correlating  these  measures  with  the  various  ARTEP  subtask 
ratings.  The  resulting  correlation  matrix  with  376  entries  contained  47 
correlations  statistically  significant  at  P<.05.  Five  of  the  outcome 
measures  accounted  for  39  of  these  significant  correlations  with  the 
remaining  nine  scattered  across  the  other  three  outcome  measures.  Since 
any  major  import  of  the  battlefield  outcome  measures  in  the  present  con¬ 
text  resides  in  those  39  correlations,  the  tabled  data  and  discussion  are 
limited  to  those  five  outcome  measures  and  the  various  ARTEP  subtasks 
with  which  they  corralate. 

The  first  column  in  Table  8  contains  the  only  negative  correlations 
in  the  Table.  For  each  of  those  four  negative  correlations,  the  inter¬ 
pretation  would  be  that  the  smaller  the  error  in  identifying  the  location 
of  the  enemy  thrust,  the  better  the  performance  on  the  respective  ARTEP 
subtasks.  This  seems  to  make  sense  given  that  three  of  the  four  ARTEP 
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casks  concern  Che  planning  and  coordinacion  of  fire  supporC  and  Che 
fourch  concerns  inCegraCing  CSS  inco  Che  scheme  of  maneuver.  Plausible 
explanadons  for  Che  posicive  correladon  of  "communicace/coordinace 
plans  and  orders"  wich  amounc  of  error  in  locacing  Che  enemy  ChrusC  are 
elusive.  In  Che  absence  of  ocher  informaclon,  Che  correladon  is  as 
likely  Co  be  spurious  as  lc  is  Co  be  aCCributable  Co  any  of  a  number  of 
remoce  explanadons. 

The  relaclon  of  enemy  losses  Co  Che  seven  ARTEP  subcasks  shown  in 
Che  second  column  of  Table  8  all  seem  reasonable.  The  beccer  one  ana¬ 
lyzes  Che  mission,  deCermlnes  criclcal  place,  organizes  for  combac, 
reinforces  Cerraln,  modifies  scheme  of  maneuver,  concencraces  and  shlfcs 
combac  power,  and  fixes  Che  syscams,  Che  greaCer  Che  losses  he  is  likely 
Co  inf lice  upon  Che  enemy. 

Mission  accompllshmenc  is  Che  ouCcome  measure  having  Che  gveacesc 
number  of  slgnlficancly  relaced  ARTEP  subcasks.  The  Chree  such  subcasks 
relacing  Co  fire  supporC  planning  again  confirm  Che  imporcance  of  chls 
area.  The  face  chac  six  subcasks  concerned  wich  preparing  and  organizing 
Che  bacdefleld  are  also  highly  relaced  Co  mission  accompllshmenc  is  noc 
surprising.  Similarly  for  Che  execuClon  porcion  of  Che  ARTEP,  dissemi- 
nadng  criclcal  combac  informaclon  and  inCelllgence  along  wich  modifying 
scheme  of  maneuver,  concenCraCing/shif cing  combac  power  and  inCegraCing 
CSS  inco  scheme  of  maneuver  were  highly  relaCed  Co  mission  accompllshmenc 
as  mlghc  be  expecCed. 


Statistically  Significant®  Correlations  with  ARTEP  Rating  for  the 
Five  Battlefield  Outcome  Measures  Having  the  Greatest  Number  of  Significant  Correlations 
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aAll  entries  are  significant  at  P<  .05  level 
^Significant  at  P<  .01  level 


Six  of  Che  seven  ARTEP  subtaaks  correlating  significantly  with  com¬ 
puter  generated  friendly  personnel  losses  also  correlate  significantly 
with  computer  generated  friendly  equipment  losses.  The  previously  men¬ 
tioned  high  correlation  between  personnel  and  equipment  losses  was  a 
precursor  of  *  nls  outcome  but  make  the  explanation  of  why  better  per¬ 
formance  on  those  ARTEP  subtasks  were  concomitants  of  greater  personnel 
and  equipment  losses  no  less  difficult.  As  proffered  earlier,  perhaps 
in  a  covering  force  mission  the  price  typically  paid  for  greater  enemy 
attrition  is  increased  friendly  attrition. 

Obviously  the  battlefield  outcome  measures  may  account  for  substan¬ 
tial  variance  in  the  performance  of  several  other  ARTEP  subtasks  which 
would  not  have  achieved  statistical  significance  with  the  present  sample 
size.  Correlations  on  the  ordkr  of  .57  to  .70  between  outcome  measures 
and  ARTEP  subtask  ratings  would  not  achieve  statistical  significance  even 
with  a  sample  size  of  10.  Nonetheless,  the  data  thus  far  are  indicative 
of  some  underlying  relationships  which  need  to  be  further  delineated  in 
subsequent  efforts  and  the  relative  importance  or  impact  of  the  various 
subtasks  on  overall  performance  more  fully  explored.  There  is  little 
reason  to  believe  the  various  subtasks  are  equally  crucial  within  any 
given  mission.  This  notion  was  grossly  examined  with  battlefield  outcomes 
as  the  criteria  by  deriving  three  summary  type  ARTEP  ratings  and  corre¬ 
lating  them  with  the  various  outcome  measures.  These  summary  ratings 
were  comprised  by  averaging  those  subtask  ratings  categorized  under  Tasks 
1  through  4  for  an  overall  planning  phase  score  and  those  under  Tasks  5 
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through  12  for  an  overall  execution  phase  score.  The  planning  and  execu¬ 
tion  scores  were  again  averaged  to  get  a  total  ARTEP  score. 

Since  there  was  no  evidence  available  to  the  contrary,  the  subtasks 
were  assumed  to  be  roughly  the  same  importance  and  hence,  were  weighted 
equally.  Not  surprisingly,  the  only  correlations  achieving  statistical 
significance,  as  seen  in  Table  9,  were  between  mission  accomplishment  and 
both  the  planning  and  total  ARTEP  ratings  and  even  here  only  about  65%  of 
the  variance  is  accounted  for.  Harking  back  to  the  data  relating  the 
individual  subtasks  to  the  outcome  measures,  it  is  apparent  that  the  sum¬ 
mary  data  are  reflecting  primarily  the  influence  of  the  13  subtasks  hav¬ 
ing  a  very  high  correlation  with  mission  accomplishment.  The  relation 
of  losses  to  the  summary  ARTEP  ratings  is  negligible  for  the  objective 
measures  and  though  substantially  higher  numerically  for  the  subjective 
estimates  of  loss  parameters,  is  still  short  of  statistical  significance. 
Thus,  it  would  seem  that  initial  indications  argue  for  differential 
weighting  of  the  subtasks  in  any  summary  score. 

Another  indicator  of  overall  command  group  performance  was  obtained 
with  the  organizational  processes  measure.  Each  separate  process  rating 
was  included  in  this  analysis  as  well  as  the  average  rating  across  all 
five  processes.  Referring  again  to  Table  9.  there  was  a  consistent 
and  highly  significant  series  of  relationships  between  the  organization 
process  ratings  and  the  subjective  assessment  of  task  force  losses, 

OPFOR  losses  and  mission  accomplishment.  The  positive  correlations 
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indicate  that  the  higher  the  rating  on  the  various  processes,  the  higher 
the  losses  suffered  by  the  opposing  forces,  the  better  the  rating  on 
mission  accomplishment,  and  (perhaps  as  a  result  of  the  underlying  rela¬ 
tionship  between  OPFOR  losses  and  task  force  losses)  the  higher  the  task 
force  losses.  Acquiring  information  seems  to  be  about  the  only  process 
dimension  which  did  not  have  a  high  positive  relation  to  mission  accom¬ 
plishment  and  OPFOR  losses.  Whether  this  is  a  function  of  inability  to 
adequately  observe  and  rate  those  activities,  a  bonafide  lack  of  corre¬ 
spondence,  or  a  spurious  result  attributable  to  the  sample  size  is  not 
known.  Again,  as  previously  noted  for  the  ARTEP  tasks,  the  objective 
outcomes  did  not  correlate  significantly  or  in  a  logically  consistent 
manner. 

The  correlations  of  overall  performance  ratings  with  battlefield 
outcomes  computed  for  each  staff  and  separate  coordinating  element,  the 
battalion  commander  and  the  command  group  as  a  whole  are  also  presented 
in  Table  9.  Of  the  five  significant  correlations,  two  were  for  the  S2 
element  -  a  positive  correlation  with  number  of  decisive  engagements  and 
a  negative  correlation  with  area  lost.  Given  the  earlier  mentioned  high 
negative  relationship  between  number  of  decisive  engagements  and  area  lost, 
this  dual  relationship  is  not  surprising.  It  does  not,  however,  make  it 
any  easier  to  understand  how,  in  a  covering  force  operation,  the  greater 
the  number  of  decisive  engagements,  the  better  the  rated  performance  of 
the  S2  element.  On  the  other  hand,  the  high  relationship  of  the  close 
air  support  element  rating  to  number  of  decisive  engagements  seems  to 
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^Significant  at  .05  level,  1  tailed  test 
**Signlf leant  at  .01  level,  1  tailed  test 
***Signif leant  at  .001  level,  1  tailed  test 
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make  more  sense,  at  least  if  viewed  as  a  function  of  Increased  opportu¬ 
nity  for  effective  performance.  The  final  significant  correlation  is 
that  between  rated  overall  performance  of  the  battalion  commander  and 
mission  accomplishment.  While  this  is  a  desired  and  reassuring  outcome, 
it  is  not  surprising  given  that  the  battalion  commander  is  normally  held 
responsible  for  the  degree  to  which  the  assigned  mission  is  accomplished. 

Experimental  Measures 

The  experimental  objective  measures  of  command  group  performance 
including  the  four  categorizations  of  communications  frequency  were  cor¬ 
related  with  battlefield  outcome  Indices  which  themselves  are  experi¬ 
mental  in  nature.  The  results  are  shown  in  Table  10.  Trends  are  diffi¬ 
cult  to  perceive  in  these  data  except  in  regard  to  locus  pf  control  and 
information  provided  to  company  commander  variables.  Significant  and 
near  significant  correlations  between  these  variables  and  enemy  thrust 
location  and  personnel  and  equipment  loss  suggest  the  following  picture. 
The  higher  (more  centralized)  the  level  of  control,  the  greater  the  error 
in  locating  the  enemy  thrust  with  a  consequent  temporary  reduction  in 
loss  of  personnel  and  equipment  for  reasons  discussed  earlier.  That 
reduction  is  concomitant  with  increasing  amounts  of  information  provided 
to  the  company  by  the  battalion  (perhaps  as  an  attempt  to  compensate 
for  perceived  deficiencies  in  the  performance  of  the  company  commander)  , 
thus,  that  Information  variable  has  a  fairly  high,  though  non-significant 
correlation  with  thrust  location  plus  significantly  high  negative  corre¬ 
lations  with  personnel  and  equipment  loss. 
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♦Significant  at  .05  level,  2  tailed  test 


Further  insights  regarding  the  experimental  measures  were  sought 
through  correlation  with  the  subjective  ratings  of  performance  on  the 
47  ARTEP  subtasks.  Fifty-eight  of  these  correlations  achieved  statisti¬ 
cal  significance.  Forty-four  of  these  involved  the  same  six  experimental 
measures.  To  promote  understanding  and  for  the  sake  of  brevity,  only 
these  44  are  presented  here  (Table  11) . 

All  eight  significant  locus  of  control/ARTE?  correlations  were  nega¬ 
tive  indicating  that  less  centralized  control  results  in  better  ARTEP 
subtask  performance.  Five  of  the  six  significant  negative  correlations 
for  "battalion  to  company  Information"  communication  involve  A?'~  0  sub¬ 
tasks  which  were  also  significantly  related  to  locus  of  control.  This 
can  be  interpreted  to  mean  that  less  communication  of  information  from 
battalion  to  company  is  associated  with  better  performance,  at  least  on 
those  ARTEP  subtasks  where  more  autonomy  is  vested  at  lower  levels. 

These  subtasks  concern  priority  of  fires  and  fire  support  coordination, 
organizing  for  combat  and  reinforcing  the  terrain,  disseminating  cr.t.-'cal 
combat  Information  and  intelligence,  and  Integrating  combat  support  serv¬ 
ices  into  the  maneuver  scheme. 

All  of  the  significant  correlations  between  ARTEP  subtasks  and  the 
"average  distance  between  battle  positions"  measure  were  negative.  This 
is  an  indication  that  at  least  for  a  covering  force  mission  of  the  type, 
in  the  location,  with  the  force  structure,  etc.,  as  played  and  evaluated 
in  this  effort,  better  performance  on  certain  ARTEP  subtasks  is  associated 
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with  smaller  average  distances  between  battle  positions.  Such  a  relation¬ 
ship  seems  quite  rational  when  one  considers  that  all  but  one  of  the  ARTEP 
subtasks  Involved  relate  to  the  execution  tasks  of  seeing  the  battlefield, 
control  and  coordination  of  combat  operations,  concentrating/ shifting 
combat  power,  securing  and  protecting  the  task  force,  and  the  special 
action  of  reacting  appropriately  to  enemy  jamaing. 

One  other  experimental  measure,  "company  to  battalion  information 
communication"  produced  all  negative  correlations  with  ARTEP  subtasks. 

All  but  one  of  these  related  to  planning  activities.  Three  of  the  seven 
concern  identification  and  gathering  critical  combat  information  and 
intelligence,  and  one  to  developing  a  communication  plan.  Why  less  com¬ 
munication  of  information  up  to  battalion  should  be  associated  with 
better  performance  on  those  ARTEP  subtasks  is  hard  to  fathom  unless  one 
assumes  that  the  rstio  of  "noise"  to  "signal"  Increases  with  the  amount 
of  information  cosmunlcatlon  from  company  to  battalion.  Though  seven  out 
of  eight  correlations  of  "company  to  battalion  requests"  with  ARTEP  sub¬ 
tasks  were  also  negative,  none  of  these  subtasks  were  the  same  as  those 
significantly  related  to  the  "company  to  battalion  information  communica¬ 
tion"  measures.  In  fact,  most  of  the  ARTEP  subtasks  significantly  related 
to  this  variable  involve  fire  support  functions  vhich  seems  to  Indicate 
that  the  less  the  company  has  to  request  Information,  the  better  these 
functions  are  performed.  Further  evidence  of  what  has  been  previously 
discussed  in  regard  to  dissemination  of  critical  combat  Information  and 
intelligence  is  reflected  in  the  negative  correlation  of  performance  on 
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TABLE  11 

Statistically  Significant*  Correlations  With  ARTEP  Ratings  for  the 
Six  Experimental  and  Communications  Measures  Having  the  Greyest  Number  of  Correlations 
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chat  subtask.  Better  performance  on  that  subtaak  is  associated  with 
less  frequent  communication  between  company  and  battalion. 

The  final  experimental  measure  for  which  significant  correlations 
were  tabled  was  number  of  battle  positions.  Three  of  these  correlations 
are  positive  and  three  are  negative.  The  three  positive  ones  relate  to 
identification  and  dissemination  of  critical  combat  information  and 
intelligence,  not  an  unreasonable  outcome  if  your  communications  are 
adquate.  The  negative  correlations  are  also  reasonable  in  that  they 
concern  providing  supplies  and  arming,  fueling  and  maintaining  the  equip¬ 
ment.  The  greater  the  number  of  battle  positions,  the  more  difficult 
such  tasks  might  be. 

CAMMS  AS  A  TRAINING  RESEARCH  VEHICLE 

There  are  a  number  of  observations.  Impressions  and  uncertainties 
generated  from  the  conduct  of  this  effort.  Most  result  from  the  actual 
process  of  planning  and  running  training  exercises  including  discussions 
with  cognizant  player  and  controller  participants.  Hard  data  of  a  form 
amenable  to  statistical  analysis  are  rarely  available  to  support  these 
findings  although  some  derive  from  inadequacies  Identified  as  part  of 
data  analysis.  The  intrinsic  value  of  these  findings  stems  from  the 
perceptions  of  the  skilled  CAMMS  team  and  their  intensive  five-week 
exposure  to  and  interaction  with  these  exercises  and  the  playing  units. 
For  purposes  of  exposition,  these  findings  are  roughly  grouped  into 
system  factors  and  application  factors.  As  a  general  prelude,  it  should 


be  stated  that  pervasive  of  both  sets  of  factors  Is  a  well  founded  con¬ 
cern  with  the  degree  to  which  performance  changes  over  time  as  assessed 
In  CAMMS  are  a  function  of  learning  how  to  better  perform  as  a  command 
group  as  opposed  to  learning  how  to  play  CAMMS  to  get  an  Improved  score. 
This  ambiguity  could  not  be  avoided  within  the  conatralnta  of  the  present 
effort  nor  would  time  and  resources  for  TEA  78  permit  a  design  from  which 
these  effects  could  be  partially  teased  out.  Essentially,  the  major  need 
Is  for  a  yardstick  or  assessment  tool  external  to  CAMMS.  It  Is  intended 
that  some  preliminary  data  bearing  on  this  question  will  be  obtained  if 
arrangements  can  be  completed  to  have  some  of  the  units  who  play  CAMMS, 
also  play  CATTS  where  any  special  skills  unique  to  the  CAMMS  vehicle  as 
opposed  to  content  will  be  discounted. 

Even  with  such  additional  data,  the  validity  of  the  simulation  and 
the  measures  of  performance  therein  will  require  further  confirmation. 
While  considerable  weapons,  tactics  and  doctrinal  experience,  expertise 
and  measurement  sophistication  have  gone  Into  developing  CAMMS,  a  really 
sound  assessment  of  the  validity  of  CAMMS  will  be  possible  only  after 
projects  currently  under  development,  e.g..  Multiple  Integrated  Laser 
Evaluation  System  and  the  National  Training  Center  r.?e  completed,  thus 
producing  a  highly  realistic  "live"  battlefie'M  trom  which  definitive 
measures  of  command  group  and  troop  performance  can  be  obtained. 

System  Factors 

Certain  characteristics  are  requisite  of  a  good  training  simulation. 
These  Include  creation  of  a  realistic  environment  within  which  events 
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unfold  and  actions  transpire,  reasonable  representation  of  all  functions 
and  tasks  which  are  deemed  Integral  and  Important  to  the  skills  being 
trained,  provisions  to  Insure  the  performance  of  these  tasks  Is  pre¬ 
cipitated,  a  means  for  credible  assessment  of  the  quality  of  performance 
and  providing  feedback  regarding  strengths  and  weaknesses  of  that  per¬ 
formance,  a  capability  to  replicate  on  essential  elements  for  equitable 
comparative  assessments  while  maintaining  sufficient  flexibility  to  pro¬ 
vide  challenge  and  opportunity  to  make  and  correct  mistakes.  A  training 
system  possessing  these  characteristics  has  all  the  Ingredients  for  sup¬ 
porting  training  effectiveness  research.  Obviously,  these  characteris¬ 
tics  are  not  absolutes.  However,  all  other  things  being  equal,  the 
better  such  requirements  or  characteristics  are  fulfilled,  the  greater 
should  be  the  possibility  of  producing  an  optimally  effective  training 
system.  Looking  at  CAMMS  from  this  perspective,  there  are  several 
observations  which  can  be  made.  The' most  general  and  perhaps  most 
Important  of  which  la  that  CAMMS  la  remarkably  well  along  on  most  of 
these  dimensions  given  Its  current  stage  of  development.  Other  more 
specific  observations  Include: 

1.  There  Is  little  basis  for  faulting  realism.  Though  unquestion¬ 
ably  greater  fidelity  could  be  achieved,  there  Is  no  evidence  of  dis¬ 
satisfaction  on  this  dimension.  In  fact,  CAMMS  was  judged  superior  to 
a  CPX  in  a  recent  survey  (Kaplan  and  Barber  -  in-press) .  Neither  Is 
there  any  evidence  that  greater  fidelity  would  beget  better  training. 


2.  To  the  extent  that  the  command  group  ARTEP  is  currently  the 
guiding  light  for  this  sort  of  training,  it  may  be  desirable  to  include 
more  extensive  admin/log  play.  The  relatively  short  duration  of  the 
execution  phases  of  the  respective  exercises  does  not  afford  much  oppor¬ 
tunity  for  play  and  evaluation  fo  the  gamut  of  these  activities.  Future 
modification  of  existing  or  generation  of  new  scenarios  should  provide 
explicit  attention  to  Insuring  that  sufficient  opportunity  for  play  of 
all  the  staff  element  functions  is  provided.  At  the  same  time,  considera¬ 
tion  should  be  given  to  whether  or  not  the  ARTEP  is  sufficiently  exhaus¬ 
tive.  It  is  suspected  that  ccmnand  group  performance  may  be  somehow 

more  than  the  sum  of  the  performances  on  the  ARTEP  tasks  and  subtasks. 

3.  A  programmed  or  scripted  set  of  proves  exceeding  substantially 
in  number  those  developed  for  this  effort  will  be  necessary  if  behaviors 
of  interest  not  normally  forthcoming  or  not  observably  explicit  in  the 
play  of  CAMMS  are  to  be  elicited  in  some  systematic  fashion.  This  is 
particularly  relevant  for  the  planning  phase  where  specific  probes  and/ 
or  situations  will  need  to  be  developed  to  elicit  the  planning  sequences 
so  that  they  may  be  observed  by  the  TOC  observers  and  controllers.  At 
the  present  time,  most  of  the  observation  takes  place  iurlng  the  delivery 
of  the  battalion  OPORD  and,  consequently,  very  Ur.le  Information  on 

the  planning  Involved  in  its  preparation  is  available. 

4.  Standards  for  the  ARTEP  subtasks  are  very  generally  worded  and 
thus,  can  apply  to  almost  any  command  group  situation.  This  generality 
leaves  much  room  for  interpretation  by  the  rater  and  consequently  can 


result  In  wide  differences  In  ratings  accorded  the  sane  group  by  differ¬ 
ent  observers.  Scoring  or  feedback  based  on  such  potentially  divergent 
judgments  Is  of  questionable  meaning  and  therefore,  Impact.  Some  elabo¬ 
ration  of  the  standards  for  specific  CAMMS  training/evaluation  exercises 
could  help.  The  same  may  be  said  of  the  personnel  and  equipment  loss 
data  calculated  by  the  computer.  The  current  sunmary  output  must  be 
re-analyzed  to  get  the  appropriate  information  for  training  feedback. 

The  re-analysls  of  the  output  to  produce  both  friendly  and  enemy  person¬ 
nel/equipment  losses  Interferes  with  the  time  needed  by  the  S1/S4  con¬ 
troller  to  perform  his  controller  functions.  Resupply  Is  usually  done 
for  the  friendly  forces  but  not  the  enemy  forces  because  the  time  to 
enter  the  resupplied  Items  Into  the  computer  would  force  the  exercise 
out  of  real  time.  In  addition,  the  summary  or  end  of  exercise  output 
makes  no  provisions  to  account  for  resupply  In  figuring  the  loss  status 
of  personnel/equipment.  Some  provision  should  be  made  either  to  Increase 
the  number  of  controller  personnel  to  enable  a  timely  re-analysis  of  the 
summary  data  or  to  devise  a  software  package  to  analyze  the  summary  data 
into  a  form  that  can  be  used  for  training  feedback  and  to  account  for 
resupply. 

5.  Standardized  elements  for  reliability  of  repeated  administrations 
is  an  elusive  target  If  you  wish  at  the  same  time  to  allow  for  player 
Innovation  and  challenging  controller  reactions,  and  to  avoid  a  rote 
learning  syndrome.  For  exportable  training  purposes  as  well  as  for 
training  research/evaluation  purposes  where  there  must  be  dependence  on 
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ad  hoc  controllers  for  many  of  the  positions,  a  programmed  text  and 
accompanying  "time  into  exercise"  reference  manual  could  provide  addi¬ 
tional  assurances  of  consistency  in  use. 

Application  Factors 

It  is  no  less  true  with  training  systems  than  with  weapons  or  other 
complex  battlefield  systems,  implementation  by  and  for  people  create 
opportunities  for  divergent  procedures  of  use.  Observations  falling 
under  the  applications  rubric  In  the  current  context  are  focused  on 
those  aspects  of  system  use  which  either  the  trainer  or  trainee  can 
capitalize  upon,  compensate  for  or  at  least  be  aware  of  In  terms  of 
possible  Impact  on  objectives. 

It  Is  often  difficult  In  a  home  station  environment  to  have  available 
on  a  continuous  basis  for  at  least  four  days  all  principal  members  of  a 
battalion  conmand  group.  Leave,  emergencies  and  other  priority  demands 
take  their  toll.  While  a  surrogate  in  an  individual  position  usually  Is 
not  all  that  debilitating,  it  can  Impact  on  the  overall  performance  of 
the  command  group  and  their  learning  rate.  On  the  other  hand,  it  can  be 
argued  that  such  turbulence  is  typical  and  therefore  training  and  evalua¬ 
tion  might  be  more  realistic  if  this  Is  allowed  to  occur.  Subsequent 
analyses  of  the  present  data  are  planned  in  an  effort  to  get  a  partial 
handle  on  the  broader  question  of  the  effects  of  command  group  turbulence 
on  performance  Insofar  as  it  appeared  as  a  variable  across  participating 
battalions.  Hence,  the  concern  is  with  length  of  tours  with  a  unit  etc., 
rather  than  presence  or  absence  for  a  training  exercise. 
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The  concern  with  player  substitution  is  paralleled  by  a  concern  by 
controller/evaluator  substitutions  occasionally  within  but  more  frequently 
between  battalions  making  comparability  tenuous  because  controller  play 
cannot  be  totally  procedurallzed  nor  evaluation  sufficiently  objectified 
to  insure  that  different  controllers  or  evaluators  reacting  to  the  same 
events  will  behave  the  same.  Consistency  is  largely  dependent  on  having 
the  same  people  from  exercise  to  exercise  within  and  battalion  to  battalion 
across  player  units.  While  failure  to  do  so  may  not  jeopardize  the  train¬ 
ing  value  obtained,  it  can  the  training  research  benefits. 

Providing  the  opportunity  and  data  to  support  feedback  to  the  playing 
unit  is  at  least  half  the  battle  toward  effective  use.  The  remaining 
hurdle  is  the  "how"  or  format  for  delivering  that  feedback.  While  only 
one  feedback  format  was  utilized,  it  appears  that  the  technique  was  much 
more  acceptable  than  some  previously  used.  The  command  groups  seemed  re¬ 
ceptive  to  the  feedback  and  found  it  informative.  The  one-on-one  feedback 
session  appears  to  be  in  a  format  that  is  more  comfortable  than  group  feed¬ 
back  sessions  for  both  the  controller  and  the  player  within  the  command 
group.  The  controllers  indicated  that  they  felt  much  more  comfortable 
and  were  much  more  willing  to  present  negative  feedback  to  the  partici¬ 
pants  in  the  one-on-one  session  than  in  group  settings. 

One  other  pertinent  aspect  of  application  concerns  the  learning  curve 
and  the  appropriate  duration  of  CAMMS  play  to  maximize  performance  improve¬ 
ment  as  a  function  of  time.  While  there  are  many  questions  in  this 
domain,  the  only  evidence  generated  in  the  present  study  suggests  that 
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performance  has  essentially  asymptoted  by  the  third  exercise  session 
(post-test) .  If  that  leveling  off  Is  attributable  to  a  learning  curve 
function,  then  more  than  two  consecutive  days  play  (at  least  of  the  same 
type  mission)  Is  not  efficient.  The  prognosis  mignt  still  be  the  same 
even  If  the  result  In  attributable  to  an  adaptation  process  where  Interest 
and  motivation  have  decreased  and  standards  have  relaxed.  Remedies  for 
the  latter  condition  might  more  easily  be  found.  Actually  the  duration 
of  a  CAMMS  exercise  or  successive  exercises,  the  amount  of  time  inter¬ 
vening  between  exercises,  how  these  should  be  Interspersed  with  use  of 
other  kinds  of  Individual  and  collective  training  systems  and  exercises 
and  many  related  questions  are  part  of  the  overall  training  strategy  the 
formulation  of  which  will  be  dependent  on  achievement  of  various  mile¬ 
stones  in  TEA  85. 


CONCLUSIONS 

The  following  conclusions  are  tempered  to  reflect  the  uncertainties 
resulting  from  constraints  In  the  conduct  of  this  effort,  most  notably, 
the  small  sample,  and  the  potential  confounding  of  measures  of  learning 
CAMMS  with  learning  that  which  is  trained  by  C AMMS : 

e  CAMMS  shows  evidence  of  being  an  eff.rtive  training  vehicle  for 
improving  battalion  command  group  proviciency  as  subjectively  judged  In 
terms  of  differential  performance  on  ARTEP  Tasks  and  Subtasks  and  an 
overall  assessment  of  the  total  command  group  and  each  of  the  major  staff 
elements.  The  generally  consistent  and  positive  changes  in  performance 


73 


across  pre-,  training  and  post-test  exercises  and  differentiation  among 
subtasks  and  elements  within  exercise  attest  to  its  utility. 

•  Development  of  a  greater  number  of  objective  measures  of  command 
group  performance  in  CAMS  is  feasible  to  both  supplement  and  ultimately 
supplant  some  of  the  existing  subjective  ratings.  It  la  a  difficult  and 
slow  process  and  should  not  be  expected  to  obviate  the  need  for  some 
subjective  ratings. 

a  The  relationship  of  consand  group  performance  to  battlefield  out¬ 
comes  is  complex.  The  quality  of  performance  on  no  single  measure  yet 
identified  can  be  adequately  Interpreted  in  isolation  of  other  measures, 
or  the  condition  of  performance. 

e  Performance  on  some  ARTZP  subtasks  appears  to  influence  battlefield 
outcomes.  Additional  effort  will  be  required  to  obtain  reliable  estimates 
of  the  degree  to  which  performance  on  other  ARTEP  subtasks  may  also  relate 
to  outcome  measures  and  what  additional  meaningful  measures  might  be 
identified  to  more  fully  reflect  the  total  dimensions  of  battlefield 
performance. 

•  Organizational  process  measures  as  used  in  this  effort  do  not 
discriminate  performance  differences  among  the  various  measures  them¬ 
selves  or  change  in  performance  as  a  function  of  the  training  exercise. 
However,  their  high  relationship  to  mission  accomplishment  and  other  out¬ 
come  measures  warrant  further  investigations. 

•  CAMMS  has  the  potential  for  fulfilling  the  requirements  of  a 
training  and  training  research  vehicle  envisaged  for  pursuit  of  TEA  85 


74 


objectives.  While  some  modifications  are  indicated,  these  are  relatively 
modest  in  nature  and  several  concern  improvements  which  would  be  made  in 
the  normal  evolution  of  CAMMS  as  resources  permitted.  The  intrinsic 
worth  and  flexibility  of  CAMMS  together  with  the  data  obtained  and  learn¬ 
ing  experienced  on  the  part  of  the  CAMMS  team  make  it  a  leading  contender 
for  this  role.  Until  other  computer  supported  developmental  simulations 
have  progressed  substantially  beyond  their  current  stage,  CAMMS  may  be 
the  only  reasonable  vehicle  for  examining  the  integration  of  troops  on 
the  ground  with  the  play  of  a  battalion  level  command  group  simulation. 
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APPENDIX  A 


General  Situation ,  Mission  Assigned,  and 
Assets  Available  to  Participating  Battalions 

GENERAL  SITUATION 

Hostilities  had  broken  out  along  the  eastern  German  border  approxi¬ 
mately  twenty-four  hours  before.  The  Park  Land  Forces  had  crossed  the 
border  and  moved  through  the  Fulc'a  Gap  area.  Their  movement  had  been 
progressively  southwestward,  with  the  intended  objective  conjectured  to 
be  Fraukfurt.  Since  the  enemy  crossed  the  national  border,  their  opposi¬ 
tion  had  been  a  corps  covering  force. 

The  XX  Armored  Division  had  been  ordered  to  establish  a  covering 
force  forward  of  the  FEBA  and  defend  In  sector  In  the  main  battle  area. 

In  turn,  the  division  had  given  parallel  missions  to  all  three  of  its 
brigades.  The  brigade's  specific  mission  was  to  establish  a  covering 
force  along  line  Gold  and  delay  In  sector  forward  of  the  FEBA,  defend 
in  zone  from  the  FEBA  to  the  brigade  rear  boundary,  retain  city  X,  and 
be  prepared  to  assist  In  the  passage  of  lines  of  the  corps  covering  force 
at  line  Delta.* 

MISSION  ASSIGNED  AND  ASSETS  AVAILABLE 

Each  of  the  three  brigade  missions  constituted  the  basis  for  estab¬ 
lishing  the  mission  for  the  pre-  and  post-test  and  training  exercises. 

*Note  that  line  Gold,  Delta,  and  the  city  to  be  retained  were  changed 
for  the  pre-  and  post- test  and  training  exercises  of  the  participating 
battalions . 
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Regardless  of  Che  specific  brigade  mission,  Che  general  mission  given 
Co  Che  parcicipacing  battalions  was  Che  same.  The  only  difference  was 
Che  cerraln  on  which  the  operation  was  conducted.  A  description  of  Che 
general  mission  of  Che  battalion  is  the  following:  establish  a  covering 
force  forward  of  the  brigade  main  battle  area,  occupy  and  prepare  various 
battle  positions  and  battle  areas,  be  prepared  to  delay  from  the  various 
battle  areas  and  battle  positions,  prepare  bridges  in  sector  for  destruc¬ 
tion,  occupy  given  battle  posi.lons  and  areas  in  MBA  upon  completion  of 
the  covering  force  mission,  and  prepare  to  reinforce  battle  areas  in 
MBA  on  order.  To  carry  out  this  mission,  each  mechanized  infantry  bat¬ 
talion  received  an  armor  company  in  exchange  for  one  of  its  companies 
and,  similarly,  armor  battalions  received  a  mechanized  Infantry  company 
in  exchange  for  one  of  their  companies.  Thus,  the  extent  of  task  organi¬ 
zation  was  controlled.  In  support  of  each  battalion,  regardless  of  type, 
was  a  platoon  of  combat  engineers.  Brigade  assets  accessible  to  the 
battalions  included  three  battalions  of  155mm  SF  howitzers,  close  air 

support,  and  attack  and  recon  helicopters  whose  use  was  under  the  control 
of  the  brigade  S3.  Divisional  GS  artillery  was  also  available  through 
the  brigade. 

As  pointed  out  above,  the  general  mission  of  the  participating 
battalions  for  all  exorcises  was  the  same;  only  the  terrain  on  which 
the  mission  was  executed  varied.  However,  the  three  battle  areas  or 
sectors  were  all  in  the  Frledberg-Frankfur.t  general  area  and  within  10 
to  25  kilometers  of  one  another.  Within  each  area  there  were  at  least 
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two  enemy  avenues  of  approach,  ard  fordable  and  non-fordable  rivers  to 
contend  with.  The  terrain,  in  general,  for  all  exercises  was  hilly 
woodland  with  flatter  farmland  interspersed  throughout.  For  the  train- 
lng  exercise,  the  terrain  was  less  wooded  and  hlll>  and  contained  more 
farmland .  There  was  also  a  greater  number  of  avenues  of  approach 


available  to  the  enemy. 
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